Jul 09 14:14:38.713514 ip-10-0-142-225 systemd[1]: Starting Kubernetes Kubelet... Jul 09 14:14:40.471759 ip-10-0-142-225 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 14:14:40.471759 ip-10-0-142-225 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 14:14:40.471759 ip-10-0-142-225 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 14:14:40.471759 ip-10-0-142-225 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 14:14:40.471759 ip-10-0-142-225 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 14:14:40.471759 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.382823 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 14:14:40.471759 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387297 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:14:40.471759 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387313 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:14:40.471759 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387318 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:14:40.471759 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387321 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:14:40.471759 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387324 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:14:40.471759 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387326 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:14:40.471759 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387330 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:14:40.471759 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387332 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:14:40.469264 ip-10-0-142-225 systemd[1]: Started Kubernetes Kubelet. Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387335 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387337 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387340 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387343 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387348 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387352 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387355 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387358 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387361 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387364 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387366 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387374 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387377 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387380 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387382 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387385 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387387 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387390 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387392 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:14:40.473775 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387396 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387398 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387402 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387404 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387407 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387410 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387412 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387415 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387418 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387420 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387423 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387425 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387428 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387430 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387433 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387435 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387439 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387443 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387447 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:14:40.476979 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387450 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387453 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387456 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387459 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387462 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387464 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387467 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387469 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387472 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387474 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387477 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387479 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387482 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387485 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387488 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387491 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387494 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387497 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387500 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387503 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:14:40.477755 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387505 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387508 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387510 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387513 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387515 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387518 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387520 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387523 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387526 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387528 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387532 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387535 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387539 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387542 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387545 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387549 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387552 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387556 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387559 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:14:40.480530 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387562 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387564 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387567 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387569 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387572 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387575 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387577 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387580 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387583 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387585 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387588 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387591 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387594 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.387597 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388354 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388362 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388365 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388368 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388371 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388374 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:14:40.481327 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388376 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388379 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388382 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388385 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388388 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388391 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388393 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388396 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388398 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388401 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388404 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388407 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388409 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388412 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388414 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388417 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388420 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388422 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388425 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388427 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:14:40.485310 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388430 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388433 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388445 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388448 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388451 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388455 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388459 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388462 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388465 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388468 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388471 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388473 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388478 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388481 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388483 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388486 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388488 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388491 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388493 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:14:40.486094 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388496 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388498 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388501 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388504 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388507 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388510 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388513 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388515 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388517 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388520 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388522 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388525 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388527 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388531 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388533 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388536 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388538 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388542 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388546 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:14:40.490140 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388549 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388551 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388554 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388557 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388559 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388562 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388564 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388567 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388570 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388572 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388575 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388578 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388581 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388583 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388586 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388589 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388592 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388595 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388597 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388600 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:14:40.490968 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388603 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388605 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388608 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388610 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388613 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388618 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388621 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388624 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388627 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388630 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388633 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388636 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388639 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388641 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.388644 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389063 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389068 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389071 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389074 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:14:40.494274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389083 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389086 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389089 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389092 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389095 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389097 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389100 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389106 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389109 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389111 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389114 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389117 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389120 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389122 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389125 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389127 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389130 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389133 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389135 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389138 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:14:40.495098 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389140 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389143 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389146 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389149 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389153 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389157 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389160 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389163 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389166 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389169 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389172 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389175 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389178 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389182 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389185 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389187 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389190 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389193 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389196 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:14:40.497995 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389200 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389202 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389205 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389208 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389211 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389214 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389217 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389219 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389222 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389225 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389230 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389232 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389235 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389237 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389240 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389242 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389245 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389247 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389250 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:14:40.498806 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389254 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389257 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389260 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389263 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389265 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389268 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389271 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389273 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389280 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389283 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389285 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389287 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389291 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389295 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389297 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389300 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389302 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389305 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389307 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389310 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:14:40.599315 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389312 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389315 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389318 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389323 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389326 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389328 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389331 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389333 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389336 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389338 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389341 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389344 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389346 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389349 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389351 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389354 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.389356 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389430 2581 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389444 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389458 2581 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389469 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 14:14:40.600107 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389477 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389485 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389491 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389496 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389500 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389504 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389507 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389511 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389514 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389517 2581 flags.go:64] FLAG: --cgroup-root="" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389520 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389523 2581 flags.go:64] FLAG: --client-ca-file="" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389526 2581 flags.go:64] FLAG: --cloud-provider="external" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389529 2581 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389535 2581 flags.go:64] FLAG: --cluster-domain="" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389538 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389541 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389545 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389548 2581 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389552 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389555 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389558 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389561 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389564 2581 flags.go:64] FLAG: --contention-profiling="false" Jul 09 14:14:40.601348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389567 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389570 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389573 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389576 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389581 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389584 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389587 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389590 2581 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389593 2581 flags.go:64] FLAG: --enable-server="true" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389596 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389604 2581 flags.go:64] FLAG: --event-burst="100" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389607 2581 flags.go:64] FLAG: --event-qps="50" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389611 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389614 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389617 2581 flags.go:64] FLAG: --eviction-hard="" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389621 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389624 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389627 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389630 2581 flags.go:64] FLAG: --eviction-soft="" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389633 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389636 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389639 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389643 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389646 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389649 2581 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 14:14:40.602126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389652 2581 flags.go:64] FLAG: --feature-gates="" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389656 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389659 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389662 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389665 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389668 2581 flags.go:64] FLAG: --healthz-port="10248" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389671 2581 flags.go:64] FLAG: --help="false" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389674 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-142-225.ec2.internal" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389677 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389680 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389683 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389687 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389690 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389693 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389696 2581 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389699 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389702 2581 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389705 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389708 2581 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389712 2581 flags.go:64] FLAG: --kube-reserved="" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389715 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389718 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389721 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389724 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 14:14:40.602927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389727 2581 flags.go:64] FLAG: --lock-file="" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389730 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389733 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389736 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389741 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389745 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389747 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389750 2581 flags.go:64] FLAG: --logging-format="text" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389753 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389756 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389759 2581 flags.go:64] FLAG: --manifest-url="" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389762 2581 flags.go:64] FLAG: --manifest-url-header="" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389766 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389770 2581 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389774 2581 flags.go:64] FLAG: --max-pods="110" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389777 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389780 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389783 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389786 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389789 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389792 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389794 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389802 2581 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389805 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389808 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 14:14:40.603970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389811 2581 flags.go:64] FLAG: --pod-cidr="" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389815 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389820 2581 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389824 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389828 2581 flags.go:64] FLAG: --pods-per-core="0" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389831 2581 flags.go:64] FLAG: --port="10250" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389834 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389837 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0efa9a510bf3552ac" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389840 2581 flags.go:64] FLAG: --qos-reserved="" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389842 2581 flags.go:64] FLAG: --read-only-port="10255" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389845 2581 flags.go:64] FLAG: --register-node="true" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389848 2581 flags.go:64] FLAG: --register-with-taints="" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389853 2581 flags.go:64] FLAG: --registry-burst="10" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389856 2581 flags.go:64] FLAG: --registry-qps="5" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389859 2581 flags.go:64] FLAG: --reserved-cpus="" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389862 2581 flags.go:64] FLAG: --reserved-memory="" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389866 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389869 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389873 2581 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389876 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389889 2581 flags.go:64] FLAG: --runonce="false" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389892 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389895 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389911 2581 flags.go:64] FLAG: --seccomp-default="false" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389915 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 14:14:40.604985 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389918 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389921 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389924 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389927 2581 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389930 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389933 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389936 2581 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389939 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389942 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389958 2581 flags.go:64] FLAG: --system-cgroups="" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389963 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389969 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389972 2581 flags.go:64] FLAG: --tls-cert-file="" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389976 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389980 2581 flags.go:64] FLAG: --tls-min-version="" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389983 2581 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389986 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389989 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389992 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.389997 2581 flags.go:64] FLAG: --v="2" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.390002 2581 flags.go:64] FLAG: --version="false" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.390006 2581 flags.go:64] FLAG: --vmodule="" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.390010 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.390013 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390138 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:14:40.606283 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390142 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390145 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390149 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390151 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390154 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390157 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390159 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390162 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390164 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390168 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390172 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390176 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390179 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390182 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390184 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390187 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390191 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390197 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390200 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:14:40.607554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390203 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390206 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390208 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390211 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390214 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390217 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390219 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390223 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390226 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390229 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390231 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390234 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390236 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390239 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390241 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390244 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390247 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390249 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390252 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:14:40.608348 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390255 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390257 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390260 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390262 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390265 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390267 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390270 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390272 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390275 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390278 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390280 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390284 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390287 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390290 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390293 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390295 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390298 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390300 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390303 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390305 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390310 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:14:40.761493 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390313 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390315 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390318 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390320 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390322 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390325 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390327 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390330 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390334 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390337 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390340 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390342 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390345 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390348 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390351 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390353 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390356 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390358 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390361 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:14:40.776897 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390363 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390365 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390368 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390372 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390374 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390378 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390381 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390383 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390386 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390388 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390391 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390393 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390397 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390400 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390402 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390405 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390407 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390410 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390412 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:14:40.781921 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.390415 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.391495 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.398543 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.398561 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.398579 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.398585 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398644 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398649 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398653 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398656 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398658 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398661 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398664 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398667 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398670 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:14:40.782731 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398673 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398675 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398678 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398681 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398683 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398686 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398689 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398691 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398694 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398697 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398700 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398702 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398705 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398708 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398710 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398713 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398715 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398718 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398720 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398722 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:14:40.885918 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398726 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398729 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398732 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398734 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398737 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398740 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398745 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398748 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398751 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398754 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398756 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398759 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398762 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398764 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398767 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398769 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398772 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398774 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398777 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:14:40.887066 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398781 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398785 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398789 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398792 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398795 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398798 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398801 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398805 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398808 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398810 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398813 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398816 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398820 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398823 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398826 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398829 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398832 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398835 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398837 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:14:40.887835 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398840 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398843 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398845 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398848 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398850 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398853 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398856 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398858 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398860 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398863 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398865 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398868 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398870 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398873 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398876 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398878 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398881 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398883 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398886 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398888 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:14:40.889376 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398894 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398896 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398899 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398901 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398904 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398907 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398910 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398912 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398915 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398918 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398920 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.398923 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.398929 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399069 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399076 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:14:40.890168 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399078 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399081 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399084 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399087 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399089 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399092 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399094 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399097 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399100 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399102 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399105 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399107 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399110 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399112 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399115 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399117 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399119 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399122 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399126 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399129 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:14:40.891311 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399132 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399134 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399137 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399142 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399144 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399147 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399151 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399153 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399156 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399158 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399161 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399163 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399166 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399168 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399172 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399175 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399177 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399180 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399182 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399184 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:14:40.892093 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399187 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399189 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399192 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399194 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399197 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399199 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399202 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399204 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399207 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399209 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399214 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399217 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399219 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399222 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399224 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399228 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399230 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399233 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399236 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399239 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:14:40.893246 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399241 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399244 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399247 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399249 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399252 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399254 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399257 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399259 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399263 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399266 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399270 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399273 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399276 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399280 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399283 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399285 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399288 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399290 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399293 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:14:40.895180 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399295 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399298 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399300 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399304 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399306 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399309 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399313 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399316 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399321 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399324 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399327 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399330 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399332 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399336 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399338 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399340 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399343 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:14:40.895694 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:40.399345 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:14:40.896135 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.399351 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 14:14:40.896135 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.399469 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 14:14:40.896135 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.405215 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 14:14:40.896135 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.406725 2581 server.go:1019] "Starting client certificate rotation" Jul 09 14:14:40.896135 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.406829 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 14:14:40.896135 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.406866 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 14:14:40.896135 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.409891 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 14:14:40.896135 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.411189 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 14:14:40.896135 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.424584 2581 log.go:25] "Validated CRI v1 runtime API" Jul 09 14:14:40.896135 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.426932 2581 log.go:25] "Validated CRI v1 image API" Jul 09 14:14:40.896135 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.427909 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 14:14:40.896135 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.435124 2581 fs.go:135] Filesystem UUIDs: map[6015c2ff-5f1f-4b90-9389-ad16a99b8789:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fa73b7ab-cc7a-434e-8252-dae328bf3027:/dev/nvme0n1p4] Jul 09 14:14:40.896457 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.435143 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 14:14:40.896457 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.439906 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.441607 2581 manager.go:216] Machine: {Timestamp:2026-07-09 14:14:40.440006529 +0000 UTC m=+0.563145903 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100121 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21b7ecbd0b5230d6a699e44a63137b SystemUUID:ec21b7ec-bd0b-5230-d6a6-99e44a63137b BootID:b56833cb-413e-42fa-bd64-72c532708c6a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5e:07:a5:25:6b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5e:07:a5:25:6b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:a5:e8:dd:67:11 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.441728 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.441857 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.445289 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.445317 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-225.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.446745 2581 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.446756 2581 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.446764 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.447523 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.448853 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.451856 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.452010 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.452040 2581 kubelet.go:486] "Attempting to sync node with API server" Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.452050 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.452068 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.452078 2581 kubelet.go:398] "Adding apiserver pod source" Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.452089 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 14:14:40.896521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.454588 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.454614 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.459556 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.461591 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.461622 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.461794 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.461811 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.461817 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.461823 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.461831 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.461837 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.462679 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.462686 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.462693 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.462699 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.462708 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.462716 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.463656 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.463664 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.468397 2581 server.go:1268] "Started kubelet" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.468485 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.468487 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.468551 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.469993 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.471168 2581 server.go:310] "Adding debug handlers to kubelet server" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.472481 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r77cl" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.600157 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r77cl" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:40.601217 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:40.602023 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-225.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:40.602169 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.602314 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-225.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:40.601979 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-225.ec2.internal.18c0a4a4df66d1c0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-225.ec2.internal,UID:ip-10-0-142-225.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-225.ec2.internal,},FirstTimestamp:2026-07-09 14:14:40.46836576 +0000 UTC m=+0.591505134,LastTimestamp:2026-07-09 14:14:40.46836576 +0000 UTC m=+0.591505134,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-225.ec2.internal,}" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.604054 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.604069 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:40.604282 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.605021 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.605058 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.605075 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.605166 2581 reconstruct.go:97] "Volume reconstruction finished" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.605210 2581 reconciler.go:29] "Reconciler: start to sync state" Jul 09 14:14:40.897033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.605414 2581 factory.go:55] Registering systemd factory Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.605430 2581 factory.go:223] Registration of the systemd container factory successfully Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.605845 2581 factory.go:153] Registering CRI-O factory Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.605860 2581 factory.go:223] Registration of the crio container factory successfully Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.605934 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.605988 2581 factory.go:103] Registering Raw factory Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.606024 2581 manager.go:1196] Started watching for new ooms in manager Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:40.606407 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-225.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:40.606539 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.606833 2581 manager.go:349] Starting recovery of all containers Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.623531 2581 manager.go:354] Recovery completed Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.627890 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.632124 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.632161 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.632172 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.632810 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.632819 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.632835 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.632842 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.633523 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.633550 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.633561 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.633989 2581 policy_none.go:49] "None policy: Start" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.634002 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.634013 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.634983 2581 policy_none.go:47] "Start" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.674564 2581 manager.go:337] "Starting Device Plugin manager" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:40.674736 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.674747 2581 server.go:87] "Starting device plugin registration server" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.674990 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.675008 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.675151 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.675247 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.675258 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:40.676260 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:40.676302 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-225.ec2.internal\" not found" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.776178 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.777698 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.777735 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.777749 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.777782 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-225.ec2.internal" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.786292 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-225.ec2.internal" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:40.786323 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-225.ec2.internal\": node \"ip-10-0-142-225.ec2.internal\" not found" Jul 09 14:14:40.898085 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:40.803692 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found" Jul 09 14:14:40.904024 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:40.903997 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found" Jul 09 14:14:40.970080 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.970040 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 14:14:40.971366 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.971345 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 14:14:40.971439 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.971389 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 14:14:40.971439 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.971421 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 14:14:40.971528 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:40.971463 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 14:14:40.973764 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:40.973738 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:14:41.004728 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:41.004696 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found" Jul 09 14:14:41.072112 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.072043 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal"] Jul 09 14:14:41.072190 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.072118 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:14:41.073797 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.073777 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:14:41.073897 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.073807 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:14:41.073897 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.073819 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:14:41.075115 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.075103 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:14:41.075229 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.075209 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.075316 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.075242 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:14:41.076707 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.076691 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:14:41.076783 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.076715 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:14:41.076783 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.076724 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:14:41.076783 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.076697 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:14:41.076783 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.076781 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:14:41.076899 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.076790 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:14:41.077922 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.077909 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.077976 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.077940 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:14:41.078684 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.078667 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:14:41.078749 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.078700 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:14:41.078749 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.078714 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:14:41.097501 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:41.097482 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-225.ec2.internal\" not found" node="ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.101011 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:41.100993 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-225.ec2.internal\" not found" node="ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.105014 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:41.104994 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found" Jul 09 14:14:41.107190 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.107177 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/570cea2ea81a2c52a69e256e396471ee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal\" (UID: \"570cea2ea81a2c52a69e256e396471ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.107247 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.107198 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/570cea2ea81a2c52a69e256e396471ee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal\" (UID: \"570cea2ea81a2c52a69e256e396471ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.107247 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.107214 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6bd8a30ab1f0a51bf21d4aa8ae4b5951-config\") pod \"kube-apiserver-proxy-ip-10-0-142-225.ec2.internal\" (UID: \"6bd8a30ab1f0a51bf21d4aa8ae4b5951\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.205763 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:41.205712 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found" Jul 09 14:14:41.207917 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.207889 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6bd8a30ab1f0a51bf21d4aa8ae4b5951-config\") pod \"kube-apiserver-proxy-ip-10-0-142-225.ec2.internal\" (UID: \"6bd8a30ab1f0a51bf21d4aa8ae4b5951\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.207917 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.207918 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/570cea2ea81a2c52a69e256e396471ee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal\" (UID: \"570cea2ea81a2c52a69e256e396471ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.208102 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.207941 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/570cea2ea81a2c52a69e256e396471ee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal\" (UID: \"570cea2ea81a2c52a69e256e396471ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.208102 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.208010 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6bd8a30ab1f0a51bf21d4aa8ae4b5951-config\") pod \"kube-apiserver-proxy-ip-10-0-142-225.ec2.internal\" (UID: \"6bd8a30ab1f0a51bf21d4aa8ae4b5951\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.208102 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.208088 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/570cea2ea81a2c52a69e256e396471ee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal\" (UID: \"570cea2ea81a2c52a69e256e396471ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.208210 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.208133 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/570cea2ea81a2c52a69e256e396471ee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal\" (UID: \"570cea2ea81a2c52a69e256e396471ee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.306231 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:41.306182 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found" Jul 09 14:14:41.399603 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.399570 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.404372 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.404348 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.406517 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:41.406495 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found" Jul 09 14:14:41.406609 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.406519 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 14:14:41.406708 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.406686 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 14:14:41.507012 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:41.506972 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found" Jul 09 14:14:41.606033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.605985 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 14:09:40 +0000 UTC" deadline="2028-03-16 16:49:50.526433868 +0000 UTC" Jul 09 14:14:41.606033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.606028 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14786h35m8.92040868s" Jul 09 14:14:41.607075 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:41.607048 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found" Jul 09 14:14:41.656148 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.656061 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:14:41.704480 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.704443 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.716855 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.716828 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 14:14:41.719134 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.719120 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal" Jul 09 14:14:41.725470 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:41.725451 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 14:14:42.100506 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:42.100467 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570cea2ea81a2c52a69e256e396471ee.slice/crio-fd6e0bdafdd8fa09089fffd9fe7cd38cfee0821bbfb5b02e85adfb40247a49ad WatchSource:0}: Error finding container fd6e0bdafdd8fa09089fffd9fe7cd38cfee0821bbfb5b02e85adfb40247a49ad: Status 404 returned error can't find the container with id fd6e0bdafdd8fa09089fffd9fe7cd38cfee0821bbfb5b02e85adfb40247a49ad Jul 09 14:14:42.106340 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.106323 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 14:14:42.141056 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:42.141029 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd8a30ab1f0a51bf21d4aa8ae4b5951.slice/crio-d7aa270552d39525013693bc71ca48a2851885844c44ad92eb338783520f1329 WatchSource:0}: Error finding container d7aa270552d39525013693bc71ca48a2851885844c44ad92eb338783520f1329: Status 404 returned error can't find the container with id d7aa270552d39525013693bc71ca48a2851885844c44ad92eb338783520f1329 Jul 09 14:14:42.188314 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.188284 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:14:42.198244 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.198218 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:14:42.453061 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.453028 2581 apiserver.go:52] "Watching apiserver" Jul 09 14:14:42.461683 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.461654 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 14:14:42.462057 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.462034 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-xj9gv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal","openshift-multus/multus-8tqrw","openshift-multus/multus-additional-cni-plugins-k6p6r","openshift-network-operator/iptables-alerter-t7cfh","kube-system/global-pull-secret-syncer-5kc7s","kube-system/konnectivity-agent-ghz28","kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal","openshift-multus/network-metrics-daemon-qr5pr","openshift-network-diagnostics/network-check-target-qqzlh","openshift-ovn-kubernetes/ovnkube-node-v67ml","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72","openshift-cluster-node-tuning-operator/tuned-xm5p8","openshift-dns/node-resolver-thcjx"] Jul 09 14:14:42.464387 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.464364 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ghz28" Jul 09 14:14:42.466636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.466615 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.467465 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.467447 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6ttcw\"" Jul 09 14:14:42.467635 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.467621 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 14:14:42.467726 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.467695 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 14:14:42.469062 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.469038 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.469641 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.469605 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 14:14:42.471978 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.470255 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rhhjg\"" Jul 09 14:14:42.471978 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.471800 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 14:14:42.472130 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.472050 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 14:14:42.472130 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.472077 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 14:14:42.472248 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.472077 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 14:14:42.472248 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.472209 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tfwcm\"" Jul 09 14:14:42.472520 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.472503 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.472843 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.472762 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:14:42.474249 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.474224 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t7cfh" Jul 09 14:14:42.475512 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.475494 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 14:14:42.475512 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.475508 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 14:14:42.475677 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.475569 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n5s5t\"" Jul 09 14:14:42.476457 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.476429 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:42.476547 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:42.476523 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:14:42.476675 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.476653 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 14:14:42.476675 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.476664 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:14:42.476868 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.476853 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 14:14:42.477266 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.477247 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jxfsv\"" Jul 09 14:14:42.478791 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.478774 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:42.478877 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:42.478841 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:14:42.481037 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.481015 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xj9gv" Jul 09 14:14:42.483204 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.483186 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-thcjx" Jul 09 14:14:42.484772 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.484751 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 14:14:42.484867 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.484793 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 14:14:42.484867 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.484832 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zzl52\"" Jul 09 14:14:42.484867 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.484837 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 14:14:42.485616 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.485600 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:42.485750 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:42.485728 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:14:42.486056 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.486007 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 14:14:42.486056 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.486008 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 14:14:42.486178 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.486091 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zl7cb\"" Jul 09 14:14:42.488012 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.487983 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.490479 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.490460 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.491475 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.491456 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 14:14:42.491562 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.491484 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 14:14:42.491562 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.491514 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5krs4\"" Jul 09 14:14:42.491995 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.491978 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 14:14:42.493058 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.493022 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 14:14:42.493058 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.493050 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 14:14:42.493212 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.493022 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 14:14:42.493919 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.493780 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 14:14:42.493919 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.493797 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 14:14:42.493919 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.493809 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fm4md\"" Jul 09 14:14:42.493919 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.493821 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 14:14:42.506207 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.506178 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 14:14:42.515972 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.515927 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ace2890b-11e9-4efe-8a64-5fc7584e7254-cni-binary-copy\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.516711 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.515980 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-sysconfig\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.516711 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516489 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-var-lib-kubelet\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.516711 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516518 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47nb7\" (UniqueName: \"kubernetes.io/projected/bd39fe24-a9ec-407b-a576-c0c1672914d7-kube-api-access-47nb7\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.516711 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516542 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4ff0d36b-2274-490a-9421-62b5a55e6028-host\") pod \"node-ca-xj9gv\" (UID: \"4ff0d36b-2274-490a-9421-62b5a55e6028\") " pod="openshift-image-registry/node-ca-xj9gv" Jul 09 14:14:42.516711 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516567 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/769bd7fc-a76c-4797-b56d-30577140294f-ovn-node-metrics-cert\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.516711 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516591 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-socket-dir\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.516711 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516611 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-hostroot\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.516711 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516656 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dsk9\" (UniqueName: \"kubernetes.io/projected/ace2890b-11e9-4efe-8a64-5fc7584e7254-kube-api-access-7dsk9\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.516711 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516688 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-sys\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.517120 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516712 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-kubelet\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.517120 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516747 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bccdd948-f9a8-410b-ab36-404e262a2989-konnectivity-ca\") pod \"konnectivity-agent-ghz28\" (UID: \"bccdd948-f9a8-410b-ab36-404e262a2989\") " pod="kube-system/konnectivity-agent-ghz28" Jul 09 14:14:42.517120 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516787 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/467732b1-3742-4e13-9647-8696ff20e359-multus-daemon-config\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.517120 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516821 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/84d37bfe-ac0f-494d-b3df-c124300fccdd-iptables-alerter-script\") pod \"iptables-alerter-t7cfh\" (UID: \"84d37bfe-ac0f-494d-b3df-c124300fccdd\") " pod="openshift-network-operator/iptables-alerter-t7cfh" Jul 09 14:14:42.517120 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516847 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-sysctl-d\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.517120 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516869 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d486c18-e090-48ce-bc55-8d752af57ea6-hosts-file\") pod \"node-resolver-thcjx\" (UID: \"4d486c18-e090-48ce-bc55-8d752af57ea6\") " pod="openshift-dns/node-resolver-thcjx" Jul 09 14:14:42.517120 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516913 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-cni-bin\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.517120 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516970 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-cni-netd\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.517120 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.516996 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-multus-conf-dir\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.517120 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517038 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:42.517120 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517079 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-run-ovn\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.517120 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517111 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-systemd-units\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517133 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-run-openvswitch\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517158 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/769bd7fc-a76c-4797-b56d-30577140294f-ovnkube-config\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517181 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517204 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-modprobe-d\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517252 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-sysctl-conf\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517273 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-etc-kubernetes\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517332 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp954\" (UniqueName: \"kubernetes.io/projected/467732b1-3742-4e13-9647-8696ff20e359-kube-api-access-kp954\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517362 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-host\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517399 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-879jp\" (UniqueName: \"kubernetes.io/projected/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-kube-api-access-879jp\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517424 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-system-cni-dir\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517448 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-multus-socket-dir-parent\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517473 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-device-dir\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517518 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-node-log\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517541 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/769bd7fc-a76c-4797-b56d-30577140294f-env-overrides\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517563 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/467732b1-3742-4e13-9647-8696ff20e359-cni-binary-copy\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.517636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517587 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-var-lib-cni-multus\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517611 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ace2890b-11e9-4efe-8a64-5fc7584e7254-os-release\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517635 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f9a1846-c394-4931-a7f5-d839e7558c70-kubelet-config\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517657 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs\") pod \"network-metrics-daemon-qr5pr\" (UID: \"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8\") " pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517722 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-etc-openvswitch\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517750 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-run-multus-certs\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517777 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-systemd\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517801 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-run\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517847 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-run-netns\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517875 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-run-ovn-kubernetes\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517901 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517931 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-cnibin\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.517985 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ace2890b-11e9-4efe-8a64-5fc7584e7254-system-cni-dir\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518010 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ace2890b-11e9-4efe-8a64-5fc7584e7254-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518033 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f9a1846-c394-4931-a7f5-d839e7558c70-dbus\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518057 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zdlh\" (UniqueName: \"kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh\") pod \"network-check-target-qqzlh\" (UID: \"66695e98-eb01-4479-bed6-81320c8b3586\") " pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:42.518385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518101 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-os-release\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518181 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-run-netns\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518206 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-var-lib-kubelet\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518237 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bw29\" (UniqueName: \"kubernetes.io/projected/84d37bfe-ac0f-494d-b3df-c124300fccdd-kube-api-access-8bw29\") pod \"iptables-alerter-t7cfh\" (UID: \"84d37bfe-ac0f-494d-b3df-c124300fccdd\") " pod="openshift-network-operator/iptables-alerter-t7cfh" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518261 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4ff0d36b-2274-490a-9421-62b5a55e6028-serviceca\") pod \"node-ca-xj9gv\" (UID: \"4ff0d36b-2274-490a-9421-62b5a55e6028\") " pod="openshift-image-registry/node-ca-xj9gv" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518290 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-etc-selinux\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518315 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ace2890b-11e9-4efe-8a64-5fc7584e7254-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518342 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-tuned\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518367 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd39fe24-a9ec-407b-a576-c0c1672914d7-tmp\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518395 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-log-socket\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518469 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/769bd7fc-a76c-4797-b56d-30577140294f-ovnkube-script-lib\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518507 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-registration-dir\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518534 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-run-k8s-cni-cncf-io\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518558 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ace2890b-11e9-4efe-8a64-5fc7584e7254-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518589 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84d37bfe-ac0f-494d-b3df-c124300fccdd-host-slash\") pod \"iptables-alerter-t7cfh\" (UID: \"84d37bfe-ac0f-494d-b3df-c124300fccdd\") " pod="openshift-network-operator/iptables-alerter-t7cfh" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518616 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-var-lib-openvswitch\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.519218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518648 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bccdd948-f9a8-410b-ab36-404e262a2989-agent-certs\") pod \"konnectivity-agent-ghz28\" (UID: \"bccdd948-f9a8-410b-ab36-404e262a2989\") " pod="kube-system/konnectivity-agent-ghz28" Jul 09 14:14:42.520041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518709 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-var-lib-cni-bin\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.520041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518744 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67sp6\" (UniqueName: \"kubernetes.io/projected/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-kube-api-access-67sp6\") pod \"network-metrics-daemon-qr5pr\" (UID: \"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8\") " pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:42.520041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518788 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-lib-modules\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.520041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518828 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzk8x\" (UniqueName: \"kubernetes.io/projected/4ff0d36b-2274-490a-9421-62b5a55e6028-kube-api-access-kzk8x\") pod \"node-ca-xj9gv\" (UID: \"4ff0d36b-2274-490a-9421-62b5a55e6028\") " pod="openshift-image-registry/node-ca-xj9gv" Jul 09 14:14:42.520041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518869 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9tpq\" (UniqueName: \"kubernetes.io/projected/4d486c18-e090-48ce-bc55-8d752af57ea6-kube-api-access-g9tpq\") pod \"node-resolver-thcjx\" (UID: \"4d486c18-e090-48ce-bc55-8d752af57ea6\") " pod="openshift-dns/node-resolver-thcjx" Jul 09 14:14:42.520041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518893 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-slash\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.520041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518925 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dlxt\" (UniqueName: \"kubernetes.io/projected/769bd7fc-a76c-4797-b56d-30577140294f-kube-api-access-8dlxt\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.520041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.518986 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-multus-cni-dir\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.520041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.519011 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ace2890b-11e9-4efe-8a64-5fc7584e7254-cnibin\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.520041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.519033 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-kubernetes\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.520041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.519065 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d486c18-e090-48ce-bc55-8d752af57ea6-tmp-dir\") pod \"node-resolver-thcjx\" (UID: \"4d486c18-e090-48ce-bc55-8d752af57ea6\") " pod="openshift-dns/node-resolver-thcjx" Jul 09 14:14:42.520041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.519087 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-run-systemd\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.520041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.519142 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-sys-fs\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.605217 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.605184 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 14:14:42.616978 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.616871 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619418 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ace2890b-11e9-4efe-8a64-5fc7584e7254-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619459 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-tuned\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619486 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd39fe24-a9ec-407b-a576-c0c1672914d7-tmp\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619509 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-log-socket\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619535 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/769bd7fc-a76c-4797-b56d-30577140294f-ovnkube-script-lib\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619558 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-registration-dir\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619582 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-run-k8s-cni-cncf-io\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619622 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ace2890b-11e9-4efe-8a64-5fc7584e7254-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619648 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84d37bfe-ac0f-494d-b3df-c124300fccdd-host-slash\") pod \"iptables-alerter-t7cfh\" (UID: \"84d37bfe-ac0f-494d-b3df-c124300fccdd\") " pod="openshift-network-operator/iptables-alerter-t7cfh" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619669 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-var-lib-openvswitch\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619690 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bccdd948-f9a8-410b-ab36-404e262a2989-agent-certs\") pod \"konnectivity-agent-ghz28\" (UID: \"bccdd948-f9a8-410b-ab36-404e262a2989\") " pod="kube-system/konnectivity-agent-ghz28" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619712 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-var-lib-cni-bin\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619735 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67sp6\" (UniqueName: \"kubernetes.io/projected/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-kube-api-access-67sp6\") pod \"network-metrics-daemon-qr5pr\" (UID: \"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8\") " pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619768 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-lib-modules\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619792 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzk8x\" (UniqueName: \"kubernetes.io/projected/4ff0d36b-2274-490a-9421-62b5a55e6028-kube-api-access-kzk8x\") pod \"node-ca-xj9gv\" (UID: \"4ff0d36b-2274-490a-9421-62b5a55e6028\") " pod="openshift-image-registry/node-ca-xj9gv" Jul 09 14:14:42.619820 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619816 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9tpq\" (UniqueName: \"kubernetes.io/projected/4d486c18-e090-48ce-bc55-8d752af57ea6-kube-api-access-g9tpq\") pod \"node-resolver-thcjx\" (UID: \"4d486c18-e090-48ce-bc55-8d752af57ea6\") " pod="openshift-dns/node-resolver-thcjx" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619861 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-slash\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619885 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dlxt\" (UniqueName: \"kubernetes.io/projected/769bd7fc-a76c-4797-b56d-30577140294f-kube-api-access-8dlxt\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619925 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-multus-cni-dir\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.619978 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ace2890b-11e9-4efe-8a64-5fc7584e7254-cnibin\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620003 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-kubernetes\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620002 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620026 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d486c18-e090-48ce-bc55-8d752af57ea6-tmp-dir\") pod \"node-resolver-thcjx\" (UID: \"4d486c18-e090-48ce-bc55-8d752af57ea6\") " pod="openshift-dns/node-resolver-thcjx" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620047 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-run-systemd\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620082 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-sys-fs\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620103 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ace2890b-11e9-4efe-8a64-5fc7584e7254-cni-binary-copy\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620124 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-sysconfig\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620147 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-var-lib-kubelet\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620170 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47nb7\" (UniqueName: \"kubernetes.io/projected/bd39fe24-a9ec-407b-a576-c0c1672914d7-kube-api-access-47nb7\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620192 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4ff0d36b-2274-490a-9421-62b5a55e6028-host\") pod \"node-ca-xj9gv\" (UID: \"4ff0d36b-2274-490a-9421-62b5a55e6028\") " pod="openshift-image-registry/node-ca-xj9gv" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620216 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/769bd7fc-a76c-4797-b56d-30577140294f-ovn-node-metrics-cert\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620245 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-socket-dir\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620265 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-hostroot\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.620829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dsk9\" (UniqueName: \"kubernetes.io/projected/ace2890b-11e9-4efe-8a64-5fc7584e7254-kube-api-access-7dsk9\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620323 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ace2890b-11e9-4efe-8a64-5fc7584e7254-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620333 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-slash\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620332 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-sys\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620381 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-sys\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620396 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-kubelet\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620445 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bccdd948-f9a8-410b-ab36-404e262a2989-konnectivity-ca\") pod \"konnectivity-agent-ghz28\" (UID: \"bccdd948-f9a8-410b-ab36-404e262a2989\") " pod="kube-system/konnectivity-agent-ghz28" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620473 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/467732b1-3742-4e13-9647-8696ff20e359-multus-daemon-config\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620499 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/84d37bfe-ac0f-494d-b3df-c124300fccdd-iptables-alerter-script\") pod \"iptables-alerter-t7cfh\" (UID: \"84d37bfe-ac0f-494d-b3df-c124300fccdd\") " pod="openshift-network-operator/iptables-alerter-t7cfh" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620524 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-sysctl-d\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620547 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d486c18-e090-48ce-bc55-8d752af57ea6-hosts-file\") pod \"node-resolver-thcjx\" (UID: \"4d486c18-e090-48ce-bc55-8d752af57ea6\") " pod="openshift-dns/node-resolver-thcjx" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620574 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-cni-bin\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620623 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-cni-netd\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620648 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-multus-conf-dir\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620714 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-multus-conf-dir\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620758 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-log-socket\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620884 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620894 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-multus-cni-dir\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.621676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620932 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-run-ovn\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620962 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-registration-dir\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620977 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-systemd-units\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.620981 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ace2890b-11e9-4efe-8a64-5fc7584e7254-cni-binary-copy\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621020 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-run-openvswitch\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621049 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/769bd7fc-a76c-4797-b56d-30577140294f-ovnkube-config\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621076 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-sysconfig\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621141 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-run-k8s-cni-cncf-io\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621155 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-sys-fs\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621190 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621200 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-var-lib-kubelet\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621209 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d486c18-e090-48ce-bc55-8d752af57ea6-tmp-dir\") pod \"node-resolver-thcjx\" (UID: \"4d486c18-e090-48ce-bc55-8d752af57ea6\") " pod="openshift-dns/node-resolver-thcjx" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:42.621284 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621337 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4ff0d36b-2274-490a-9421-62b5a55e6028-host\") pod \"node-ca-xj9gv\" (UID: \"4ff0d36b-2274-490a-9421-62b5a55e6028\") " pod="openshift-image-registry/node-ca-xj9gv" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:42.621361 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret podName:6f9a1846-c394-4931-a7f5-d839e7558c70 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:43.121338547 +0000 UTC m=+3.244477932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret") pod "global-pull-secret-syncer-5kc7s" (UID: "6f9a1846-c394-4931-a7f5-d839e7558c70") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621400 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-run-ovn\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621413 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/769bd7fc-a76c-4797-b56d-30577140294f-ovnkube-script-lib\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.622508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621449 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-socket-dir\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621491 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ace2890b-11e9-4efe-8a64-5fc7584e7254-cnibin\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621502 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-kubernetes\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621513 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-sysctl-d\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621532 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/84d37bfe-ac0f-494d-b3df-c124300fccdd-iptables-alerter-script\") pod \"iptables-alerter-t7cfh\" (UID: \"84d37bfe-ac0f-494d-b3df-c124300fccdd\") " pod="openshift-network-operator/iptables-alerter-t7cfh" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621552 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-hostroot\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621552 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-kubelet\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621549 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-modprobe-d\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621602 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-sysctl-conf\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621598 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d486c18-e090-48ce-bc55-8d752af57ea6-hosts-file\") pod \"node-resolver-thcjx\" (UID: \"4d486c18-e090-48ce-bc55-8d752af57ea6\") " pod="openshift-dns/node-resolver-thcjx" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621632 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-etc-kubernetes\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621655 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-cni-bin\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621657 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp954\" (UniqueName: \"kubernetes.io/projected/467732b1-3742-4e13-9647-8696ff20e359-kube-api-access-kp954\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621667 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ace2890b-11e9-4efe-8a64-5fc7584e7254-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621681 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-host\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621696 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-cni-netd\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621724 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-host\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.623336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621739 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-var-lib-openvswitch\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621776 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84d37bfe-ac0f-494d-b3df-c124300fccdd-host-slash\") pod \"iptables-alerter-t7cfh\" (UID: \"84d37bfe-ac0f-494d-b3df-c124300fccdd\") " pod="openshift-network-operator/iptables-alerter-t7cfh" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621793 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-var-lib-cni-bin\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621021 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-systemd-units\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621815 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-run-openvswitch\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621052 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-run-systemd\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622038 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-lib-modules\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622058 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-modprobe-d\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622082 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-etc-kubernetes\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622126 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bccdd948-f9a8-410b-ab36-404e262a2989-konnectivity-ca\") pod \"konnectivity-agent-ghz28\" (UID: \"bccdd948-f9a8-410b-ab36-404e262a2989\") " pod="kube-system/konnectivity-agent-ghz28" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622191 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-sysctl-conf\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622421 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/769bd7fc-a76c-4797-b56d-30577140294f-ovnkube-config\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.621724 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-879jp\" (UniqueName: \"kubernetes.io/projected/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-kube-api-access-879jp\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622590 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-system-cni-dir\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622623 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-multus-socket-dir-parent\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622654 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-device-dir\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622695 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-node-log\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622711 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-system-cni-dir\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.624180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622725 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/769bd7fc-a76c-4797-b56d-30577140294f-env-overrides\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622748 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/467732b1-3742-4e13-9647-8696ff20e359-cni-binary-copy\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622785 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-device-dir\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622787 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-var-lib-cni-multus\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622830 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-var-lib-cni-multus\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622849 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ace2890b-11e9-4efe-8a64-5fc7584e7254-os-release\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622879 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f9a1846-c394-4931-a7f5-d839e7558c70-kubelet-config\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622905 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs\") pod \"network-metrics-daemon-qr5pr\" (UID: \"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8\") " pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622931 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-etc-openvswitch\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622976 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-run-multus-certs\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623004 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-systemd\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623020 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f9a1846-c394-4931-a7f5-d839e7558c70-kubelet-config\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623027 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-run\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623049 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-run-netns\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623075 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-run-ovn-kubernetes\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623079 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-multus-socket-dir-parent\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623108 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.624896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623137 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-cnibin\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623142 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ace2890b-11e9-4efe-8a64-5fc7584e7254-os-release\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623160 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ace2890b-11e9-4efe-8a64-5fc7584e7254-system-cni-dir\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623190 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-systemd\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.622879 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-node-log\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ace2890b-11e9-4efe-8a64-5fc7584e7254-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623266 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/769bd7fc-a76c-4797-b56d-30577140294f-env-overrides\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:42.623277 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623302 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f9a1846-c394-4931-a7f5-d839e7558c70-dbus\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623324 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/467732b1-3742-4e13-9647-8696ff20e359-multus-daemon-config\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:42.623350 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs podName:4f5c80d5-b98a-41b3-b68a-94e8f1b303d8 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:43.12333386 +0000 UTC m=+3.246473237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs") pod "network-metrics-daemon-qr5pr" (UID: "4f5c80d5-b98a-41b3-b68a-94e8f1b303d8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623374 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zdlh\" (UniqueName: \"kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh\") pod \"network-check-target-qqzlh\" (UID: \"66695e98-eb01-4479-bed6-81320c8b3586\") " pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623381 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-etc-openvswitch\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623419 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-run-multus-certs\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623442 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-os-release\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623468 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-run-netns\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623472 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f9a1846-c394-4931-a7f5-d839e7558c70-dbus\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:42.625554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623495 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-var-lib-kubelet\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623543 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-var-lib-kubelet\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623602 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-os-release\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623629 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bw29\" (UniqueName: \"kubernetes.io/projected/84d37bfe-ac0f-494d-b3df-c124300fccdd-kube-api-access-8bw29\") pod \"iptables-alerter-t7cfh\" (UID: \"84d37bfe-ac0f-494d-b3df-c124300fccdd\") " pod="openshift-network-operator/iptables-alerter-t7cfh" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623654 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4ff0d36b-2274-490a-9421-62b5a55e6028-serviceca\") pod \"node-ca-xj9gv\" (UID: \"4ff0d36b-2274-490a-9421-62b5a55e6028\") " pod="openshift-image-registry/node-ca-xj9gv" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623690 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-etc-selinux\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623729 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ace2890b-11e9-4efe-8a64-5fc7584e7254-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623791 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd39fe24-a9ec-407b-a576-c0c1672914d7-run\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623801 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-host-run-netns\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623794 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-run-netns\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623737 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/467732b1-3742-4e13-9647-8696ff20e359-cni-binary-copy\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623834 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ace2890b-11e9-4efe-8a64-5fc7584e7254-system-cni-dir\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623869 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623877 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-etc-selinux\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623904 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/467732b1-3742-4e13-9647-8696ff20e359-cnibin\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.623941 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/769bd7fc-a76c-4797-b56d-30577140294f-host-run-ovn-kubernetes\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.624110 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd39fe24-a9ec-407b-a576-c0c1672914d7-tmp\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.624322 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4ff0d36b-2274-490a-9421-62b5a55e6028-serviceca\") pod \"node-ca-xj9gv\" (UID: \"4ff0d36b-2274-490a-9421-62b5a55e6028\") " pod="openshift-image-registry/node-ca-xj9gv" Jul 09 14:14:42.626285 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.624346 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd39fe24-a9ec-407b-a576-c0c1672914d7-etc-tuned\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.627104 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.625559 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bccdd948-f9a8-410b-ab36-404e262a2989-agent-certs\") pod \"konnectivity-agent-ghz28\" (UID: \"bccdd948-f9a8-410b-ab36-404e262a2989\") " pod="kube-system/konnectivity-agent-ghz28" Jul 09 14:14:42.627104 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.625761 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/769bd7fc-a76c-4797-b56d-30577140294f-ovn-node-metrics-cert\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.630240 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.629737 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47nb7\" (UniqueName: \"kubernetes.io/projected/bd39fe24-a9ec-407b-a576-c0c1672914d7-kube-api-access-47nb7\") pod \"tuned-xm5p8\" (UID: \"bd39fe24-a9ec-407b-a576-c0c1672914d7\") " pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.630240 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:42.630220 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:14:42.630240 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:42.630243 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:14:42.631034 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:42.630255 2581 projected.go:196] Error preparing data for projected volume kube-api-access-4zdlh for pod openshift-network-diagnostics/network-check-target-qqzlh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:14:42.631034 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:42.630352 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh podName:66695e98-eb01-4479-bed6-81320c8b3586 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:43.130331388 +0000 UTC m=+3.253470769 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4zdlh" (UniqueName: "kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh") pod "network-check-target-qqzlh" (UID: "66695e98-eb01-4479-bed6-81320c8b3586") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:14:42.631034 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.630479 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dlxt\" (UniqueName: \"kubernetes.io/projected/769bd7fc-a76c-4797-b56d-30577140294f-kube-api-access-8dlxt\") pod \"ovnkube-node-v67ml\" (UID: \"769bd7fc-a76c-4797-b56d-30577140294f\") " pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.632179 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.631728 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzk8x\" (UniqueName: \"kubernetes.io/projected/4ff0d36b-2274-490a-9421-62b5a55e6028-kube-api-access-kzk8x\") pod \"node-ca-xj9gv\" (UID: \"4ff0d36b-2274-490a-9421-62b5a55e6028\") " pod="openshift-image-registry/node-ca-xj9gv" Jul 09 14:14:42.632992 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.632889 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bw29\" (UniqueName: \"kubernetes.io/projected/84d37bfe-ac0f-494d-b3df-c124300fccdd-kube-api-access-8bw29\") pod \"iptables-alerter-t7cfh\" (UID: \"84d37bfe-ac0f-494d-b3df-c124300fccdd\") " pod="openshift-network-operator/iptables-alerter-t7cfh" Jul 09 14:14:42.633130 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.633103 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp954\" (UniqueName: \"kubernetes.io/projected/467732b1-3742-4e13-9647-8696ff20e359-kube-api-access-kp954\") pod \"multus-8tqrw\" (UID: \"467732b1-3742-4e13-9647-8696ff20e359\") " pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.634440 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.633979 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dsk9\" (UniqueName: \"kubernetes.io/projected/ace2890b-11e9-4efe-8a64-5fc7584e7254-kube-api-access-7dsk9\") pod \"multus-additional-cni-plugins-k6p6r\" (UID: \"ace2890b-11e9-4efe-8a64-5fc7584e7254\") " pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.634440 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.634398 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67sp6\" (UniqueName: \"kubernetes.io/projected/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-kube-api-access-67sp6\") pod \"network-metrics-daemon-qr5pr\" (UID: \"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8\") " pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:42.635078 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.635057 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9tpq\" (UniqueName: \"kubernetes.io/projected/4d486c18-e090-48ce-bc55-8d752af57ea6-kube-api-access-g9tpq\") pod \"node-resolver-thcjx\" (UID: \"4d486c18-e090-48ce-bc55-8d752af57ea6\") " pod="openshift-dns/node-resolver-thcjx" Jul 09 14:14:42.635265 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.635245 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-879jp\" (UniqueName: \"kubernetes.io/projected/004ce9dd-5e4e-4f92-b7af-d19a5709ef70-kube-api-access-879jp\") pod \"aws-ebs-csi-driver-node-f8f72\" (UID: \"004ce9dd-5e4e-4f92-b7af-d19a5709ef70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.647348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.647315 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2b6p2" Jul 09 14:14:42.653518 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.653455 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2b6p2" Jul 09 14:14:42.776893 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.776810 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ghz28" Jul 09 14:14:42.784839 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.784809 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" Jul 09 14:14:42.788798 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:42.788770 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbccdd948_f9a8_410b_ab36_404e262a2989.slice/crio-89349b9b053e4185395297e37370a9e08ec9236d32f13eca93ea211b49b4d4fc WatchSource:0}: Error finding container 89349b9b053e4185395297e37370a9e08ec9236d32f13eca93ea211b49b4d4fc: Status 404 returned error can't find the container with id 89349b9b053e4185395297e37370a9e08ec9236d32f13eca93ea211b49b4d4fc Jul 09 14:14:42.793724 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.793699 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8tqrw" Jul 09 14:14:42.794428 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:42.794406 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd39fe24_a9ec_407b_a576_c0c1672914d7.slice/crio-473a652342ab1008d8e26a19eaabc946acd8b685979a2a431226a12333ef7b6b WatchSource:0}: Error finding container 473a652342ab1008d8e26a19eaabc946acd8b685979a2a431226a12333ef7b6b: Status 404 returned error can't find the container with id 473a652342ab1008d8e26a19eaabc946acd8b685979a2a431226a12333ef7b6b Jul 09 14:14:42.800014 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.799996 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k6p6r" Jul 09 14:14:42.802279 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:42.802250 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod467732b1_3742_4e13_9647_8696ff20e359.slice/crio-8e3f02f3f10dbd8cb29fdd25ca5b0e8b59e204c468425e95505568b62baaa154 WatchSource:0}: Error finding container 8e3f02f3f10dbd8cb29fdd25ca5b0e8b59e204c468425e95505568b62baaa154: Status 404 returned error can't find the container with id 8e3f02f3f10dbd8cb29fdd25ca5b0e8b59e204c468425e95505568b62baaa154 Jul 09 14:14:42.806926 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.806904 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t7cfh" Jul 09 14:14:42.809684 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:42.809658 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace2890b_11e9_4efe_8a64_5fc7584e7254.slice/crio-71e4e15014372a268d4985f85915c21664eb7eeb918f0dd4b093d5b507fdcc41 WatchSource:0}: Error finding container 71e4e15014372a268d4985f85915c21664eb7eeb918f0dd4b093d5b507fdcc41: Status 404 returned error can't find the container with id 71e4e15014372a268d4985f85915c21664eb7eeb918f0dd4b093d5b507fdcc41 Jul 09 14:14:42.814032 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.814005 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xj9gv" Jul 09 14:14:42.816524 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:42.816500 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84d37bfe_ac0f_494d_b3df_c124300fccdd.slice/crio-a6dbe033224502361ac2b4f3267857821b5266ac3a34b8c7b7aa5cac1fa03b9e WatchSource:0}: Error finding container a6dbe033224502361ac2b4f3267857821b5266ac3a34b8c7b7aa5cac1fa03b9e: Status 404 returned error can't find the container with id a6dbe033224502361ac2b4f3267857821b5266ac3a34b8c7b7aa5cac1fa03b9e Jul 09 14:14:42.823615 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.823591 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-thcjx" Jul 09 14:14:42.824517 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:42.824481 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ff0d36b_2274_490a_9421_62b5a55e6028.slice/crio-1db35e328efa0cb9e380e947bb09e1238cd5359d2a07a55acd48bdd0e9e2ef40 WatchSource:0}: Error finding container 1db35e328efa0cb9e380e947bb09e1238cd5359d2a07a55acd48bdd0e9e2ef40: Status 404 returned error can't find the container with id 1db35e328efa0cb9e380e947bb09e1238cd5359d2a07a55acd48bdd0e9e2ef40 Jul 09 14:14:42.832220 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.832198 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:14:42.832893 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:42.832870 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d486c18_e090_48ce_bc55_8d752af57ea6.slice/crio-9abf5b4898c4b664235ef3aefdecfc21d6c970e9dc650da75f3d1e5b91a14f43 WatchSource:0}: Error finding container 9abf5b4898c4b664235ef3aefdecfc21d6c970e9dc650da75f3d1e5b91a14f43: Status 404 returned error can't find the container with id 9abf5b4898c4b664235ef3aefdecfc21d6c970e9dc650da75f3d1e5b91a14f43 Jul 09 14:14:42.838490 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.838466 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" Jul 09 14:14:42.841459 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:42.841322 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod769bd7fc_a76c_4797_b56d_30577140294f.slice/crio-1f137d461494d57a9c277cf6aeb00d77e80e7fd16e593a95e458421ab1ebfb86 WatchSource:0}: Error finding container 1f137d461494d57a9c277cf6aeb00d77e80e7fd16e593a95e458421ab1ebfb86: Status 404 returned error can't find the container with id 1f137d461494d57a9c277cf6aeb00d77e80e7fd16e593a95e458421ab1ebfb86 Jul 09 14:14:42.848237 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:14:42.848208 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod004ce9dd_5e4e_4f92_b7af_d19a5709ef70.slice/crio-3ece627c68d2633b1535fc2c1d21fc79b4048c1a27a01c9b119fb384049f9648 WatchSource:0}: Error finding container 3ece627c68d2633b1535fc2c1d21fc79b4048c1a27a01c9b119fb384049f9648: Status 404 returned error can't find the container with id 3ece627c68d2633b1535fc2c1d21fc79b4048c1a27a01c9b119fb384049f9648 Jul 09 14:14:42.918699 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.918635 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:14:42.977570 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.977506 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" event={"ID":"004ce9dd-5e4e-4f92-b7af-d19a5709ef70","Type":"ContainerStarted","Data":"3ece627c68d2633b1535fc2c1d21fc79b4048c1a27a01c9b119fb384049f9648"} Jul 09 14:14:42.979005 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.978976 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" event={"ID":"769bd7fc-a76c-4797-b56d-30577140294f","Type":"ContainerStarted","Data":"1f137d461494d57a9c277cf6aeb00d77e80e7fd16e593a95e458421ab1ebfb86"} Jul 09 14:14:42.980221 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.980181 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-thcjx" event={"ID":"4d486c18-e090-48ce-bc55-8d752af57ea6","Type":"ContainerStarted","Data":"9abf5b4898c4b664235ef3aefdecfc21d6c970e9dc650da75f3d1e5b91a14f43"} Jul 09 14:14:42.981573 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.981522 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ghz28" event={"ID":"bccdd948-f9a8-410b-ab36-404e262a2989","Type":"ContainerStarted","Data":"89349b9b053e4185395297e37370a9e08ec9236d32f13eca93ea211b49b4d4fc"} Jul 09 14:14:42.982931 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.982897 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal" event={"ID":"6bd8a30ab1f0a51bf21d4aa8ae4b5951","Type":"ContainerStarted","Data":"d7aa270552d39525013693bc71ca48a2851885844c44ad92eb338783520f1329"} Jul 09 14:14:42.984179 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.984146 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" event={"ID":"570cea2ea81a2c52a69e256e396471ee","Type":"ContainerStarted","Data":"fd6e0bdafdd8fa09089fffd9fe7cd38cfee0821bbfb5b02e85adfb40247a49ad"} Jul 09 14:14:42.985518 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.985492 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xj9gv" event={"ID":"4ff0d36b-2274-490a-9421-62b5a55e6028","Type":"ContainerStarted","Data":"1db35e328efa0cb9e380e947bb09e1238cd5359d2a07a55acd48bdd0e9e2ef40"} Jul 09 14:14:42.986761 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.986732 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t7cfh" event={"ID":"84d37bfe-ac0f-494d-b3df-c124300fccdd","Type":"ContainerStarted","Data":"a6dbe033224502361ac2b4f3267857821b5266ac3a34b8c7b7aa5cac1fa03b9e"} Jul 09 14:14:42.988007 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.987980 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k6p6r" event={"ID":"ace2890b-11e9-4efe-8a64-5fc7584e7254","Type":"ContainerStarted","Data":"71e4e15014372a268d4985f85915c21664eb7eeb918f0dd4b093d5b507fdcc41"} Jul 09 14:14:42.989286 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.989261 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8tqrw" event={"ID":"467732b1-3742-4e13-9647-8696ff20e359","Type":"ContainerStarted","Data":"8e3f02f3f10dbd8cb29fdd25ca5b0e8b59e204c468425e95505568b62baaa154"} Jul 09 14:14:42.990612 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:42.990587 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" event={"ID":"bd39fe24-a9ec-407b-a576-c0c1672914d7","Type":"ContainerStarted","Data":"473a652342ab1008d8e26a19eaabc946acd8b685979a2a431226a12333ef7b6b"} Jul 09 14:14:43.126862 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:43.126821 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:43.127135 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:43.126883 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs\") pod \"network-metrics-daemon-qr5pr\" (UID: \"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8\") " pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:43.127135 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:43.127020 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:14:43.127135 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:43.127054 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:14:43.127135 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:43.127104 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret podName:6f9a1846-c394-4931-a7f5-d839e7558c70 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:44.127083241 +0000 UTC m=+4.250222622 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret") pod "global-pull-secret-syncer-5kc7s" (UID: "6f9a1846-c394-4931-a7f5-d839e7558c70") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:14:43.127135 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:43.127126 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs podName:4f5c80d5-b98a-41b3-b68a-94e8f1b303d8 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:44.127115543 +0000 UTC m=+4.250254931 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs") pod "network-metrics-daemon-qr5pr" (UID: "4f5c80d5-b98a-41b3-b68a-94e8f1b303d8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:14:43.228070 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:43.228031 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zdlh\" (UniqueName: \"kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh\") pod \"network-check-target-qqzlh\" (UID: \"66695e98-eb01-4479-bed6-81320c8b3586\") " pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:43.228256 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:43.228234 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:14:43.228328 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:43.228260 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:14:43.228328 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:43.228270 2581 projected.go:196] Error preparing data for projected volume kube-api-access-4zdlh for pod openshift-network-diagnostics/network-check-target-qqzlh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:14:43.228431 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:43.228336 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh podName:66695e98-eb01-4479-bed6-81320c8b3586 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:44.228316596 +0000 UTC m=+4.351455980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4zdlh" (UniqueName: "kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh") pod "network-check-target-qqzlh" (UID: "66695e98-eb01-4479-bed6-81320c8b3586") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:14:43.659251 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:43.659191 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 14:09:42 +0000 UTC" deadline="2027-12-31 08:18:56.820937336 +0000 UTC" Jul 09 14:14:43.659251 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:43.659251 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12954h4m13.161692777s" Jul 09 14:14:43.974446 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:43.974362 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:43.974606 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:43.974502 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:14:43.974606 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:43.974596 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:43.974718 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:43.974672 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:14:43.974779 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:43.974740 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:43.974829 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:43.974816 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:14:44.135315 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:44.135279 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:44.135502 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:44.135339 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs\") pod \"network-metrics-daemon-qr5pr\" (UID: \"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8\") " pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:44.135502 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:44.135466 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:14:44.135610 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:44.135531 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs podName:4f5c80d5-b98a-41b3-b68a-94e8f1b303d8 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:46.135512043 +0000 UTC m=+6.258651408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs") pod "network-metrics-daemon-qr5pr" (UID: "4f5c80d5-b98a-41b3-b68a-94e8f1b303d8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:14:44.135819 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:44.135800 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:14:44.135898 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:44.135862 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret podName:6f9a1846-c394-4931-a7f5-d839e7558c70 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:46.135842837 +0000 UTC m=+6.258982208 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret") pod "global-pull-secret-syncer-5kc7s" (UID: "6f9a1846-c394-4931-a7f5-d839e7558c70") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:14:44.235994 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:44.235883 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zdlh\" (UniqueName: \"kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh\") pod \"network-check-target-qqzlh\" (UID: \"66695e98-eb01-4479-bed6-81320c8b3586\") " pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:44.236176 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:44.236095 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:14:44.236176 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:44.236120 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:14:44.236176 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:44.236134 2581 projected.go:196] Error preparing data for projected volume kube-api-access-4zdlh for pod openshift-network-diagnostics/network-check-target-qqzlh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:14:44.236332 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:44.236195 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh podName:66695e98-eb01-4479-bed6-81320c8b3586 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:46.23617683 +0000 UTC m=+6.359316209 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4zdlh" (UniqueName: "kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh") pod "network-check-target-qqzlh" (UID: "66695e98-eb01-4479-bed6-81320c8b3586") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:14:44.659449 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:44.659399 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 14:09:42 +0000 UTC" deadline="2027-12-27 01:37:38.477252076 +0000 UTC" Jul 09 14:14:44.659449 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:44.659446 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12851h22m53.817810863s" Jul 09 14:14:45.971784 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:45.971744 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:45.972303 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:45.971888 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:14:45.972303 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:45.972248 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:45.972415 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:45.972320 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:14:45.972415 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:45.972374 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:45.972539 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:45.972468 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:14:46.155868 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:46.155063 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:46.155868 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:46.155132 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs\") pod \"network-metrics-daemon-qr5pr\" (UID: \"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8\") " pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:46.155868 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:46.155309 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:14:46.155868 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:46.155402 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs podName:4f5c80d5-b98a-41b3-b68a-94e8f1b303d8 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:50.1553823 +0000 UTC m=+10.278521675 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs") pod "network-metrics-daemon-qr5pr" (UID: "4f5c80d5-b98a-41b3-b68a-94e8f1b303d8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:14:46.155868 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:46.155467 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:14:46.155868 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:46.155500 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret podName:6f9a1846-c394-4931-a7f5-d839e7558c70 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:50.155489825 +0000 UTC m=+10.278629188 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret") pod "global-pull-secret-syncer-5kc7s" (UID: "6f9a1846-c394-4931-a7f5-d839e7558c70") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:14:46.256936 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:46.256188 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zdlh\" (UniqueName: \"kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh\") pod \"network-check-target-qqzlh\" (UID: \"66695e98-eb01-4479-bed6-81320c8b3586\") " pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:46.256936 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:46.256424 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:14:46.256936 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:46.256456 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:14:46.256936 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:46.256471 2581 projected.go:196] Error preparing data for projected volume kube-api-access-4zdlh for pod openshift-network-diagnostics/network-check-target-qqzlh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:14:46.256936 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:46.256533 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh podName:66695e98-eb01-4479-bed6-81320c8b3586 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:50.256513299 +0000 UTC m=+10.379652677 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4zdlh" (UniqueName: "kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh") pod "network-check-target-qqzlh" (UID: "66695e98-eb01-4479-bed6-81320c8b3586") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:14:47.972081 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:47.971988 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:47.972530 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:47.972132 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:14:47.972603 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:47.972546 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:47.972603 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:47.972577 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:47.972703 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:47.972650 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:14:47.972750 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:47.972732 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:14:49.972619 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:49.972582 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:49.973101 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:49.972729 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:14:49.973101 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:49.972765 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:49.973101 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:49.972884 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:14:49.973101 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:49.972587 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:49.973101 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:49.973015 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:14:50.190972 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:50.190755 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:50.190972 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:50.190815 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs\") pod \"network-metrics-daemon-qr5pr\" (UID: \"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8\") " pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:50.190972 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:50.190912 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:14:50.190972 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:50.190912 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:14:50.191301 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:50.190991 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs podName:4f5c80d5-b98a-41b3-b68a-94e8f1b303d8 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:58.190971221 +0000 UTC m=+18.314110582 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs") pod "network-metrics-daemon-qr5pr" (UID: "4f5c80d5-b98a-41b3-b68a-94e8f1b303d8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:14:50.191301 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:50.191061 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret podName:6f9a1846-c394-4931-a7f5-d839e7558c70 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:58.191026638 +0000 UTC m=+18.314166001 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret") pod "global-pull-secret-syncer-5kc7s" (UID: "6f9a1846-c394-4931-a7f5-d839e7558c70") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:14:50.291218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:50.291132 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zdlh\" (UniqueName: \"kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh\") pod \"network-check-target-qqzlh\" (UID: \"66695e98-eb01-4479-bed6-81320c8b3586\") " pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:50.291379 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:50.291361 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:14:50.291436 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:50.291388 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:14:50.291436 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:50.291401 2581 projected.go:196] Error preparing data for projected volume kube-api-access-4zdlh for pod openshift-network-diagnostics/network-check-target-qqzlh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:14:50.291524 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:50.291462 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh podName:66695e98-eb01-4479-bed6-81320c8b3586 nodeName:}" failed. No retries permitted until 2026-07-09 14:14:58.291444362 +0000 UTC m=+18.414583726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4zdlh" (UniqueName: "kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh") pod "network-check-target-qqzlh" (UID: "66695e98-eb01-4479-bed6-81320c8b3586") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:14:51.972348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:51.971809 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:51.972348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:51.971848 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:51.972348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:51.971848 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:51.972348 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:51.971987 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:14:51.972348 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:51.972054 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:14:51.972348 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:51.972146 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:14:53.971809 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:53.971771 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:53.972284 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:53.971813 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:53.972284 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:53.971879 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:53.972284 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:53.971895 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:14:53.972284 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:53.972022 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:14:53.972284 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:53.972115 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:14:55.972351 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:55.972315 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:55.972351 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:55.972353 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:55.972839 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:55.972358 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:55.972839 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:55.972440 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:14:55.972839 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:55.972548 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:14:55.972839 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:55.972657 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:14:57.972593 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:57.972548 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:57.973110 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:57.972676 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:14:57.973110 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:57.972561 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:57.973110 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:57.972743 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:57.973110 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:57.972867 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:14:57.973110 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:57.972962 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:14:58.255071 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:58.254970 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:58.255071 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:58.255036 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs\") pod \"network-metrics-daemon-qr5pr\" (UID: \"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8\") " pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:58.255305 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:58.255150 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:14:58.255305 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:58.255165 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:14:58.255305 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:58.255232 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs podName:4f5c80d5-b98a-41b3-b68a-94e8f1b303d8 nodeName:}" failed. No retries permitted until 2026-07-09 14:15:14.255211124 +0000 UTC m=+34.378350494 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs") pod "network-metrics-daemon-qr5pr" (UID: "4f5c80d5-b98a-41b3-b68a-94e8f1b303d8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:14:58.255305 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:58.255252 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret podName:6f9a1846-c394-4931-a7f5-d839e7558c70 nodeName:}" failed. No retries permitted until 2026-07-09 14:15:14.255243104 +0000 UTC m=+34.378382479 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret") pod "global-pull-secret-syncer-5kc7s" (UID: "6f9a1846-c394-4931-a7f5-d839e7558c70") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:14:58.355369 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:58.355333 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zdlh\" (UniqueName: \"kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh\") pod \"network-check-target-qqzlh\" (UID: \"66695e98-eb01-4479-bed6-81320c8b3586\") " pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:58.355550 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:58.355491 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:14:58.355550 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:58.355509 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:14:58.355550 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:58.355518 2581 projected.go:196] Error preparing data for projected volume kube-api-access-4zdlh for pod openshift-network-diagnostics/network-check-target-qqzlh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:14:58.355712 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:58.355574 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh podName:66695e98-eb01-4479-bed6-81320c8b3586 nodeName:}" failed. No retries permitted until 2026-07-09 14:15:14.355556157 +0000 UTC m=+34.478695519 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4zdlh" (UniqueName: "kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh") pod "network-check-target-qqzlh" (UID: "66695e98-eb01-4479-bed6-81320c8b3586") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:14:59.972572 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:59.972386 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:14:59.973031 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:59.972386 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:14:59.973031 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:59.972665 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:14:59.973031 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:14:59.972386 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:14:59.973031 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:59.972729 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:14:59.973031 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:14:59.972844 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:15:01.972004 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:01.971870 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:15:01.972389 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:01.972024 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:15:01.972389 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:01.972111 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:15:01.972389 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:01.972210 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:15:01.972389 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:01.972264 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:15:01.972389 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:01.972339 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:15:02.072798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:02.072766 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal" event={"ID":"6bd8a30ab1f0a51bf21d4aa8ae4b5951","Type":"ContainerStarted","Data":"48dddee28f294cff3b5e50c1a3d2e39897129a98f29fb194227778fcbdb51072"} Jul 09 14:15:02.086522 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:02.086452 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" podStartSLOduration=3.038994049 podStartE2EDuration="22.086429252s" totalImagesPullingTime="19.047435203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:14:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:02.085710558 +0000 UTC m=+22.208849942" watchObservedRunningTime="2026-07-09 14:15:02.086429252 +0000 UTC m=+22.209568644" Jul 09 14:15:02.101347 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:02.101288 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal" podStartSLOduration=21.101274527 podStartE2EDuration="21.101274527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:14:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:02.100413498 +0000 UTC m=+22.223552896" watchObservedRunningTime="2026-07-09 14:15:02.101274527 +0000 UTC m=+22.224413910" Jul 09 14:15:03.077362 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.076996 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:15:03.078151 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.077617 2581 generic.go:356] "Generic (PLEG): container finished" podID="769bd7fc-a76c-4797-b56d-30577140294f" containerID="93cfb032310de73c24de707456656b64da431cb1168f39fb1bee1ca7c25e421f" exitCode=1 Jul 09 14:15:03.078151 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.077676 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" event={"ID":"769bd7fc-a76c-4797-b56d-30577140294f","Type":"ContainerStarted","Data":"60fa645080685fd4ff411f6cbb844be91023999ba2ae805abe1ee57dc174ed88"} Jul 09 14:15:03.078151 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.077741 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" event={"ID":"769bd7fc-a76c-4797-b56d-30577140294f","Type":"ContainerStarted","Data":"cd09db9c84e28fbd4393ade866762c8a52b591add4fd48640154ed49e9fc88df"} Jul 09 14:15:03.078151 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.077786 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" event={"ID":"769bd7fc-a76c-4797-b56d-30577140294f","Type":"ContainerStarted","Data":"0543f8bbef73ca2707405086a94fe0090e04a9894b04399670ae871895b64795"} Jul 09 14:15:03.078151 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.077824 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" event={"ID":"769bd7fc-a76c-4797-b56d-30577140294f","Type":"ContainerStarted","Data":"f7b2214548ae3b432bef9ff0dbbe8eacd32c3d9c2a071b8469d2ea982ed753c6"} Jul 09 14:15:03.078151 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.077868 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" event={"ID":"769bd7fc-a76c-4797-b56d-30577140294f","Type":"ContainerDied","Data":"93cfb032310de73c24de707456656b64da431cb1168f39fb1bee1ca7c25e421f"} Jul 09 14:15:03.078151 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.077904 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" event={"ID":"769bd7fc-a76c-4797-b56d-30577140294f","Type":"ContainerStarted","Data":"aabd8acc3119bd4470be5b489a94a5da4971c169e157a44f60f07bff329f9907"} Jul 09 14:15:03.079172 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.079141 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-thcjx" event={"ID":"4d486c18-e090-48ce-bc55-8d752af57ea6","Type":"ContainerStarted","Data":"9b7fcf199b5b8eb31858b560f2fcfd66e369f4b36eadc4a471842b9c26e58db9"} Jul 09 14:15:03.080581 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.080559 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ghz28" event={"ID":"bccdd948-f9a8-410b-ab36-404e262a2989","Type":"ContainerStarted","Data":"a921da1666b25c0abf7e2d1895f85dbebee44c6aa07c45420056be8e568e1dd0"} Jul 09 14:15:03.081927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.081907 2581 generic.go:356] "Generic (PLEG): container finished" podID="570cea2ea81a2c52a69e256e396471ee" containerID="87bd0c7f74494518f1377c43afbc52d2befb54e01bce4d436561c88543559990" exitCode=0 Jul 09 14:15:03.082027 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.081979 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" event={"ID":"570cea2ea81a2c52a69e256e396471ee","Type":"ContainerDied","Data":"87bd0c7f74494518f1377c43afbc52d2befb54e01bce4d436561c88543559990"} Jul 09 14:15:03.083320 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.083295 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xj9gv" event={"ID":"4ff0d36b-2274-490a-9421-62b5a55e6028","Type":"ContainerStarted","Data":"46f06dac7c81b917af3b6143f6ad53ca755d9751c2c46cb082322c86217a8e65"} Jul 09 14:15:03.086102 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.086076 2581 generic.go:356] "Generic (PLEG): container finished" podID="ace2890b-11e9-4efe-8a64-5fc7584e7254" containerID="7aa1f046aea35eede721b57d9014d18504b2b4cb2a0f22d3e5b8b72bda29fd12" exitCode=0 Jul 09 14:15:03.086192 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.086160 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k6p6r" event={"ID":"ace2890b-11e9-4efe-8a64-5fc7584e7254","Type":"ContainerDied","Data":"7aa1f046aea35eede721b57d9014d18504b2b4cb2a0f22d3e5b8b72bda29fd12"} Jul 09 14:15:03.087685 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.087661 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8tqrw" event={"ID":"467732b1-3742-4e13-9647-8696ff20e359","Type":"ContainerStarted","Data":"2d67899b7455d8bf29a67658d3d546caa6298dcb0570adb767bebf0ff261292c"} Jul 09 14:15:03.089208 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.089184 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xm5p8" event={"ID":"bd39fe24-a9ec-407b-a576-c0c1672914d7","Type":"ContainerStarted","Data":"3244d7df82aa1c19b1c35eb87a60f6be03535cd74229f1a7aba24f9840a0db23"} Jul 09 14:15:03.090603 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.090583 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" event={"ID":"004ce9dd-5e4e-4f92-b7af-d19a5709ef70","Type":"ContainerStarted","Data":"0a2c0422bc4c2e00e16c293df6a9305f6cf90bddd4e5a93121598c0b38843235"} Jul 09 14:15:03.095374 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.095338 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-thcjx" podStartSLOduration=4.086853118 podStartE2EDuration="23.095326965s" totalImagesPullingTime="19.008473847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:14:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:03.094899302 +0000 UTC m=+23.218038685" watchObservedRunningTime="2026-07-09 14:15:03.095326965 +0000 UTC m=+23.218466349" Jul 09 14:15:03.154534 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.154488 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xj9gv" podStartSLOduration=4.13427585 podStartE2EDuration="23.15447277s" totalImagesPullingTime="19.02019692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:14:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:03.154262157 +0000 UTC m=+23.277401539" watchObservedRunningTime="2026-07-09 14:15:03.15447277 +0000 UTC m=+23.277612153" Jul 09 14:15:03.171301 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.171244 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ghz28" podStartSLOduration=17.874453929 podStartE2EDuration="23.171222763s" totalImagesPullingTime="5.296768834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:14:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:03.170226932 +0000 UTC m=+23.293366334" watchObservedRunningTime="2026-07-09 14:15:03.171222763 +0000 UTC m=+23.294362148" Jul 09 14:15:03.189588 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.188961 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8tqrw" podStartSLOduration=4.131608952 podStartE2EDuration="23.18893093s" totalImagesPullingTime="19.057321978s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:14:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:03.188741704 +0000 UTC m=+23.311881089" watchObservedRunningTime="2026-07-09 14:15:03.18893093 +0000 UTC m=+23.312070313" Jul 09 14:15:03.972133 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.972091 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:15:03.972133 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.972108 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:15:03.972446 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:03.972089 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:15:03.972446 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:03.972217 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:15:03.972446 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:03.972264 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:15:03.972446 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:03.972355 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:15:04.094622 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:04.094586 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" event={"ID":"570cea2ea81a2c52a69e256e396471ee","Type":"ContainerStarted","Data":"f3778eb44870bb909b635a03f6ce5cfcda359d49628cb97eeb95e2b8e29f87e7"} Jul 09 14:15:04.099014 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:04.098963 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t7cfh" event={"ID":"84d37bfe-ac0f-494d-b3df-c124300fccdd","Type":"ContainerStarted","Data":"f38dbc136815a946e43592b1a1520af1ebb3bbaf2d5b951603ac95c83ef6f20c"} Jul 09 14:15:04.112556 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:04.112499 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" podStartSLOduration=23.112479705 podStartE2EDuration="23.112479705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:14:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:04.112125332 +0000 UTC m=+24.235264716" watchObservedRunningTime="2026-07-09 14:15:04.112479705 +0000 UTC m=+24.235619091" Jul 09 14:15:04.130022 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:04.129964 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-t7cfh" podStartSLOduration=5.105265121 podStartE2EDuration="24.129915066s" totalImagesPullingTime="19.024649945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:14:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:04.129175489 +0000 UTC m=+24.252314871" watchObservedRunningTime="2026-07-09 14:15:04.129915066 +0000 UTC m=+24.253054449" Jul 09 14:15:04.254238 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:04.254190 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 14:15:04.690447 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:04.690327 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T14:15:04.254215732Z","UUID":"add2ed6e-95ad-450a-9885-aee6d690604e","Handler":null,"Name":"","Endpoint":""} Jul 09 14:15:04.692090 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:04.692068 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 14:15:04.692212 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:04.692113 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 14:15:04.903099 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:04.903065 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ghz28" Jul 09 14:15:04.903697 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:04.903674 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ghz28" Jul 09 14:15:05.103339 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:05.103298 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" event={"ID":"004ce9dd-5e4e-4f92-b7af-d19a5709ef70","Type":"ContainerStarted","Data":"637e48d7203bd8640e72abae71afca61bc68e64b70ada3fa125fb69fc6dd3225"} Jul 09 14:15:05.106987 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:05.106963 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:15:05.107440 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:05.107407 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" event={"ID":"769bd7fc-a76c-4797-b56d-30577140294f","Type":"ContainerStarted","Data":"5db6eb95bbc005d2c916f91de0390d154ddc81622d3e9115e25598fbd375c9f3"} Jul 09 14:15:05.971705 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:05.971626 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:15:05.971705 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:05.971652 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:15:05.971928 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:05.971642 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:15:05.971928 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:05.971760 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:15:05.971928 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:05.971901 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:15:05.972097 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:05.971984 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:15:06.086886 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:06.086855 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ghz28" Jul 09 14:15:06.087735 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:06.087708 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ghz28" Jul 09 14:15:06.111989 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:06.111932 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" event={"ID":"004ce9dd-5e4e-4f92-b7af-d19a5709ef70","Type":"ContainerStarted","Data":"d1ebc4dc4d790a427b7624ecb0dc04a774be0b7ff49f3cce3bb1471bca154ed2"} Jul 09 14:15:06.135014 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:06.134963 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f8f72" podStartSLOduration=3.875023406 podStartE2EDuration="26.134932691s" totalImagesPullingTime="22.259909285s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:14:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:06.134369732 +0000 UTC m=+26.257509144" watchObservedRunningTime="2026-07-09 14:15:06.134932691 +0000 UTC m=+26.258072075" Jul 09 14:15:07.972425 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:07.971915 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:15:07.973097 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:07.971936 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:15:07.973097 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:07.972556 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:15:07.973097 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:07.971936 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:15:07.973097 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:07.972614 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:15:07.973097 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:07.972689 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:15:08.117581 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:08.117551 2581 generic.go:356] "Generic (PLEG): container finished" podID="ace2890b-11e9-4efe-8a64-5fc7584e7254" containerID="3de1962a86890146e3e8cc7e9b360dcaceb29efa4af6dbc4feb884a82ca420e5" exitCode=0 Jul 09 14:15:08.117767 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:08.117648 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k6p6r" event={"ID":"ace2890b-11e9-4efe-8a64-5fc7584e7254","Type":"ContainerDied","Data":"3de1962a86890146e3e8cc7e9b360dcaceb29efa4af6dbc4feb884a82ca420e5"} Jul 09 14:15:08.121237 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:08.121219 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:15:08.121622 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:08.121601 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" event={"ID":"769bd7fc-a76c-4797-b56d-30577140294f","Type":"ContainerStarted","Data":"e9898ca95853726c07f2bfe4174580740bf983a954b6b64926adbb7b5967887e"} Jul 09 14:15:08.121850 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:08.121833 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:15:08.121923 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:08.121911 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:15:08.122047 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:08.122034 2581 scope.go:117] "RemoveContainer" containerID="93cfb032310de73c24de707456656b64da431cb1168f39fb1bee1ca7c25e421f" Jul 09 14:15:08.139395 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:08.139374 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:15:09.127295 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:09.127262 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:15:09.127759 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:09.127640 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" event={"ID":"769bd7fc-a76c-4797-b56d-30577140294f","Type":"ContainerStarted","Data":"37b366c0b0e83581a907ea85467ea8a439fd808f866f46711b442ff0ace13c52"} Jul 09 14:15:09.127988 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:09.127965 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:15:09.142738 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:09.142714 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:15:09.169166 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:09.169119 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" podStartSLOduration=10.033097193 podStartE2EDuration="29.169102779s" totalImagesPullingTime="19.136005586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:14:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:09.167696565 +0000 UTC m=+29.290835948" watchObservedRunningTime="2026-07-09 14:15:09.169102779 +0000 UTC m=+29.292242143" Jul 09 14:15:09.779799 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:09.779768 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qr5pr"] Jul 09 14:15:09.780010 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:09.779894 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:15:09.780010 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:09.780002 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:15:09.782697 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:09.782666 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5kc7s"] Jul 09 14:15:09.782811 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:09.782795 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:15:09.782915 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:09.782897 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:15:09.795522 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:09.795483 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qqzlh"] Jul 09 14:15:09.795676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:09.795620 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:15:09.795752 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:09.795728 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:15:10.975987 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:10.974668 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:15:10.975987 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:10.974807 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:15:10.975987 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:10.975293 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:15:10.975987 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:10.975424 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:15:11.972695 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:11.972480 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:15:11.972847 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:11.972730 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:15:12.136212 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:12.136181 2581 generic.go:356] "Generic (PLEG): container finished" podID="ace2890b-11e9-4efe-8a64-5fc7584e7254" containerID="58159668ef27610a79494194d897b30afbb2866c36d17c2293092e64582a2ebe" exitCode=0 Jul 09 14:15:12.136566 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:12.136223 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k6p6r" event={"ID":"ace2890b-11e9-4efe-8a64-5fc7584e7254","Type":"ContainerDied","Data":"58159668ef27610a79494194d897b30afbb2866c36d17c2293092e64582a2ebe"} Jul 09 14:15:12.972047 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:12.972021 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:15:12.972184 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:12.972131 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5kc7s" podUID="6f9a1846-c394-4931-a7f5-d839e7558c70" Jul 09 14:15:12.972249 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:12.972204 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:15:12.972303 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:12.972287 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qqzlh" podUID="66695e98-eb01-4479-bed6-81320c8b3586" Jul 09 14:15:13.140823 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:13.140729 2581 generic.go:356] "Generic (PLEG): container finished" podID="ace2890b-11e9-4efe-8a64-5fc7584e7254" containerID="6559fc3c6dd366000f27ff2b3e92f99dea3673cd8afc40e218addc73655bab8c" exitCode=0 Jul 09 14:15:13.140823 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:13.140779 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k6p6r" event={"ID":"ace2890b-11e9-4efe-8a64-5fc7584e7254","Type":"ContainerDied","Data":"6559fc3c6dd366000f27ff2b3e92f99dea3673cd8afc40e218addc73655bab8c"} Jul 09 14:15:13.972732 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:13.972691 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:15:13.973013 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:13.972848 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qr5pr" podUID="4f5c80d5-b98a-41b3-b68a-94e8f1b303d8" Jul 09 14:15:14.264692 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.264599 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs\") pod \"network-metrics-daemon-qr5pr\" (UID: \"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8\") " pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:15:14.265185 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:14.264780 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:15:14.265185 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.264783 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:15:14.265185 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:14.264848 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:15:14.265185 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:14.264856 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs podName:4f5c80d5-b98a-41b3-b68a-94e8f1b303d8 nodeName:}" failed. No retries permitted until 2026-07-09 14:15:46.264836238 +0000 UTC m=+66.387975621 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs") pod "network-metrics-daemon-qr5pr" (UID: "4f5c80d5-b98a-41b3-b68a-94e8f1b303d8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:15:14.265185 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:14.264931 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret podName:6f9a1846-c394-4931-a7f5-d839e7558c70 nodeName:}" failed. No retries permitted until 2026-07-09 14:15:46.264910879 +0000 UTC m=+66.388050253 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret") pod "global-pull-secret-syncer-5kc7s" (UID: "6f9a1846-c394-4931-a7f5-d839e7558c70") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:15:14.366192 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.366153 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zdlh\" (UniqueName: \"kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh\") pod \"network-check-target-qqzlh\" (UID: \"66695e98-eb01-4479-bed6-81320c8b3586\") " pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:15:14.366355 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:14.366303 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:15:14.366355 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:14.366318 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:15:14.366355 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:14.366331 2581 projected.go:196] Error preparing data for projected volume kube-api-access-4zdlh for pod openshift-network-diagnostics/network-check-target-qqzlh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:15:14.366490 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:14.366386 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh podName:66695e98-eb01-4479-bed6-81320c8b3586 nodeName:}" failed. No retries permitted until 2026-07-09 14:15:46.366372057 +0000 UTC m=+66.489511419 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4zdlh" (UniqueName: "kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh") pod "network-check-target-qqzlh" (UID: "66695e98-eb01-4479-bed6-81320c8b3586") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:15:14.792399 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.792367 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeReady" Jul 09 14:15:14.792591 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.792552 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 14:15:14.828661 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.828625 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9f8cc8f45-kvxn7"] Jul 09 14:15:14.831817 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.831792 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:14.834669 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.834643 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 14:15:14.834815 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.834735 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 14:15:14.834881 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.834846 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-57dt4\"" Jul 09 14:15:14.835044 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.835024 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 14:15:14.842332 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.842304 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 14:15:14.845861 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.845836 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9f8cc8f45-kvxn7"] Jul 09 14:15:14.849778 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.849755 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kksh7"] Jul 09 14:15:14.852649 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.852626 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9zzdz"] Jul 09 14:15:14.852794 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.852776 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kksh7" Jul 09 14:15:14.855602 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.855580 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:14.855782 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.855765 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 14:15:14.856106 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.856087 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4fkhx\"" Jul 09 14:15:14.856197 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.856132 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 14:15:14.856197 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.856095 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 14:15:14.858443 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.858424 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 14:15:14.858637 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.858618 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 14:15:14.858637 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.858633 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dj2q5\"" Jul 09 14:15:14.858875 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.858646 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 14:15:14.859155 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.859128 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 14:15:14.864141 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.864121 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kksh7"] Jul 09 14:15:14.865028 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.865003 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9zzdz"] Jul 09 14:15:14.943468 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.943430 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mpgmz"] Jul 09 14:15:14.946478 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.946457 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:14.949756 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.949730 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 14:15:14.950032 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.950009 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kggvr\"" Jul 09 14:15:14.950162 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.950069 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 14:15:14.957324 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.957302 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mpgmz"] Jul 09 14:15:14.970733 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.970705 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/741c6816-f1eb-478a-a1f2-f77d86cf08d3-ca-trust-extracted\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:14.970859 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.970742 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/741c6816-f1eb-478a-a1f2-f77d86cf08d3-registry-certificates\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:14.970859 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.970771 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/741c6816-f1eb-478a-a1f2-f77d86cf08d3-bound-sa-token\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:14.970859 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.970831 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldks9\" (UniqueName: \"kubernetes.io/projected/a3216d15-42b7-49e7-b5b8-01f35726265b-kube-api-access-ldks9\") pod \"ingress-canary-kksh7\" (UID: \"a3216d15-42b7-49e7-b5b8-01f35726265b\") " pod="openshift-ingress-canary/ingress-canary-kksh7" Jul 09 14:15:14.971035 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.970878 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/741c6816-f1eb-478a-a1f2-f77d86cf08d3-image-registry-private-configuration\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:14.971035 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.970909 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/741c6816-f1eb-478a-a1f2-f77d86cf08d3-registry-tls\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:14.971035 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.970935 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4551179-366a-4b3a-a07e-12e4d2d32994-crio-socket\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:14.971035 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.971017 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/741c6816-f1eb-478a-a1f2-f77d86cf08d3-installation-pull-secrets\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:14.971233 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.971079 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/741c6816-f1eb-478a-a1f2-f77d86cf08d3-trusted-ca\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:14.971233 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.971110 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlc7z\" (UniqueName: \"kubernetes.io/projected/a4551179-366a-4b3a-a07e-12e4d2d32994-kube-api-access-tlc7z\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:14.971233 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.971137 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4551179-366a-4b3a-a07e-12e4d2d32994-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:14.971233 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.971167 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4551179-366a-4b3a-a07e-12e4d2d32994-data-volume\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:14.971428 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.971233 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4551179-366a-4b3a-a07e-12e4d2d32994-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:14.971428 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.971267 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3216d15-42b7-49e7-b5b8-01f35726265b-cert\") pod \"ingress-canary-kksh7\" (UID: \"a3216d15-42b7-49e7-b5b8-01f35726265b\") " pod="openshift-ingress-canary/ingress-canary-kksh7" Jul 09 14:15:14.971428 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.971312 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8xf9\" (UniqueName: \"kubernetes.io/projected/741c6816-f1eb-478a-a1f2-f77d86cf08d3-kube-api-access-z8xf9\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:14.971696 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.971677 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:15:14.971813 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.971710 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:15:14.975429 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.975410 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wzd4v\"" Jul 09 14:15:14.975659 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.975644 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 14:15:14.975732 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.975683 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 14:15:14.975963 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:14.975933 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 14:15:15.072086 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.071986 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/741c6816-f1eb-478a-a1f2-f77d86cf08d3-ca-trust-extracted\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.072252 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072094 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/741c6816-f1eb-478a-a1f2-f77d86cf08d3-registry-certificates\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.072252 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072121 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/741c6816-f1eb-478a-a1f2-f77d86cf08d3-bound-sa-token\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.072252 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072146 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldks9\" (UniqueName: \"kubernetes.io/projected/a3216d15-42b7-49e7-b5b8-01f35726265b-kube-api-access-ldks9\") pod \"ingress-canary-kksh7\" (UID: \"a3216d15-42b7-49e7-b5b8-01f35726265b\") " pod="openshift-ingress-canary/ingress-canary-kksh7" Jul 09 14:15:15.072252 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072191 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/741c6816-f1eb-478a-a1f2-f77d86cf08d3-image-registry-private-configuration\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.072252 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072213 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/741c6816-f1eb-478a-a1f2-f77d86cf08d3-registry-tls\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.072252 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072233 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4551179-366a-4b3a-a07e-12e4d2d32994-crio-socket\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:15.072573 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072262 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/741c6816-f1eb-478a-a1f2-f77d86cf08d3-installation-pull-secrets\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.072573 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072292 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/741c6816-f1eb-478a-a1f2-f77d86cf08d3-trusted-ca\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.072573 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072310 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a0409b2-5366-4b47-9fd8-1f893c1d43d0-config-volume\") pod \"dns-default-mpgmz\" (UID: \"9a0409b2-5366-4b47-9fd8-1f893c1d43d0\") " pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:15.072573 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072351 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a0409b2-5366-4b47-9fd8-1f893c1d43d0-metrics-tls\") pod \"dns-default-mpgmz\" (UID: \"9a0409b2-5366-4b47-9fd8-1f893c1d43d0\") " pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:15.072573 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072378 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlc7z\" (UniqueName: \"kubernetes.io/projected/a4551179-366a-4b3a-a07e-12e4d2d32994-kube-api-access-tlc7z\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:15.072573 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072395 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4551179-366a-4b3a-a07e-12e4d2d32994-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:15.072573 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072420 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4551179-366a-4b3a-a07e-12e4d2d32994-data-volume\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:15.072573 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072471 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4551179-366a-4b3a-a07e-12e4d2d32994-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:15.072573 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072472 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/741c6816-f1eb-478a-a1f2-f77d86cf08d3-ca-trust-extracted\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.072573 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072495 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a0409b2-5366-4b47-9fd8-1f893c1d43d0-tmp-dir\") pod \"dns-default-mpgmz\" (UID: \"9a0409b2-5366-4b47-9fd8-1f893c1d43d0\") " pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:15.073109 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072720 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4551179-366a-4b3a-a07e-12e4d2d32994-crio-socket\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:15.073109 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.072946 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/741c6816-f1eb-478a-a1f2-f77d86cf08d3-registry-certificates\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.073192 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.073118 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3216d15-42b7-49e7-b5b8-01f35726265b-cert\") pod \"ingress-canary-kksh7\" (UID: \"a3216d15-42b7-49e7-b5b8-01f35726265b\") " pod="openshift-ingress-canary/ingress-canary-kksh7" Jul 09 14:15:15.073192 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.073150 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtcnk\" (UniqueName: \"kubernetes.io/projected/9a0409b2-5366-4b47-9fd8-1f893c1d43d0-kube-api-access-jtcnk\") pod \"dns-default-mpgmz\" (UID: \"9a0409b2-5366-4b47-9fd8-1f893c1d43d0\") " pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:15.073273 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.073198 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8xf9\" (UniqueName: \"kubernetes.io/projected/741c6816-f1eb-478a-a1f2-f77d86cf08d3-kube-api-access-z8xf9\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.073316 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.073274 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4551179-366a-4b3a-a07e-12e4d2d32994-data-volume\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:15.073393 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.073370 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/741c6816-f1eb-478a-a1f2-f77d86cf08d3-trusted-ca\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.073608 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.073564 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4551179-366a-4b3a-a07e-12e4d2d32994-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:15.077195 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.077171 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/741c6816-f1eb-478a-a1f2-f77d86cf08d3-installation-pull-secrets\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.077195 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.077189 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/741c6816-f1eb-478a-a1f2-f77d86cf08d3-image-registry-private-configuration\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.077378 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.077277 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/741c6816-f1eb-478a-a1f2-f77d86cf08d3-registry-tls\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.077378 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.077307 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4551179-366a-4b3a-a07e-12e4d2d32994-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:15.077788 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.077763 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3216d15-42b7-49e7-b5b8-01f35726265b-cert\") pod \"ingress-canary-kksh7\" (UID: \"a3216d15-42b7-49e7-b5b8-01f35726265b\") " pod="openshift-ingress-canary/ingress-canary-kksh7" Jul 09 14:15:15.083450 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.083427 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldks9\" (UniqueName: \"kubernetes.io/projected/a3216d15-42b7-49e7-b5b8-01f35726265b-kube-api-access-ldks9\") pod \"ingress-canary-kksh7\" (UID: \"a3216d15-42b7-49e7-b5b8-01f35726265b\") " pod="openshift-ingress-canary/ingress-canary-kksh7" Jul 09 14:15:15.083563 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.083426 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlc7z\" (UniqueName: \"kubernetes.io/projected/a4551179-366a-4b3a-a07e-12e4d2d32994-kube-api-access-tlc7z\") pod \"insights-runtime-extractor-9zzdz\" (UID: \"a4551179-366a-4b3a-a07e-12e4d2d32994\") " pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:15.083563 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.083536 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8xf9\" (UniqueName: \"kubernetes.io/projected/741c6816-f1eb-478a-a1f2-f77d86cf08d3-kube-api-access-z8xf9\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.083641 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.083571 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/741c6816-f1eb-478a-a1f2-f77d86cf08d3-bound-sa-token\") pod \"image-registry-9f8cc8f45-kvxn7\" (UID: \"741c6816-f1eb-478a-a1f2-f77d86cf08d3\") " pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.143575 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.143540 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:15.165214 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.164908 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kksh7" Jul 09 14:15:15.172530 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.172347 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9zzdz" Jul 09 14:15:15.173618 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.173586 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a0409b2-5366-4b47-9fd8-1f893c1d43d0-tmp-dir\") pod \"dns-default-mpgmz\" (UID: \"9a0409b2-5366-4b47-9fd8-1f893c1d43d0\") " pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:15.173825 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.173625 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtcnk\" (UniqueName: \"kubernetes.io/projected/9a0409b2-5366-4b47-9fd8-1f893c1d43d0-kube-api-access-jtcnk\") pod \"dns-default-mpgmz\" (UID: \"9a0409b2-5366-4b47-9fd8-1f893c1d43d0\") " pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:15.173825 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.173708 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a0409b2-5366-4b47-9fd8-1f893c1d43d0-config-volume\") pod \"dns-default-mpgmz\" (UID: \"9a0409b2-5366-4b47-9fd8-1f893c1d43d0\") " pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:15.173825 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.173731 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a0409b2-5366-4b47-9fd8-1f893c1d43d0-metrics-tls\") pod \"dns-default-mpgmz\" (UID: \"9a0409b2-5366-4b47-9fd8-1f893c1d43d0\") " pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:15.174618 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.174541 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a0409b2-5366-4b47-9fd8-1f893c1d43d0-tmp-dir\") pod \"dns-default-mpgmz\" (UID: \"9a0409b2-5366-4b47-9fd8-1f893c1d43d0\") " pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:15.174834 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.174810 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a0409b2-5366-4b47-9fd8-1f893c1d43d0-config-volume\") pod \"dns-default-mpgmz\" (UID: \"9a0409b2-5366-4b47-9fd8-1f893c1d43d0\") " pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:15.176521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.176391 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a0409b2-5366-4b47-9fd8-1f893c1d43d0-metrics-tls\") pod \"dns-default-mpgmz\" (UID: \"9a0409b2-5366-4b47-9fd8-1f893c1d43d0\") " pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:15.184691 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.184639 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtcnk\" (UniqueName: \"kubernetes.io/projected/9a0409b2-5366-4b47-9fd8-1f893c1d43d0-kube-api-access-jtcnk\") pod \"dns-default-mpgmz\" (UID: \"9a0409b2-5366-4b47-9fd8-1f893c1d43d0\") " pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:15.257363 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.256917 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:15.336377 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.335981 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kksh7"] Jul 09 14:15:15.356081 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.354227 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9f8cc8f45-kvxn7"] Jul 09 14:15:15.379286 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:15:15.379253 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod741c6816_f1eb_478a_a1f2_f77d86cf08d3.slice/crio-d92e4c886a7180be2421b08083f8751e6b38994686e942b46f76a21d09330fcb WatchSource:0}: Error finding container d92e4c886a7180be2421b08083f8751e6b38994686e942b46f76a21d09330fcb: Status 404 returned error can't find the container with id d92e4c886a7180be2421b08083f8751e6b38994686e942b46f76a21d09330fcb Jul 09 14:15:15.388633 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.388559 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9zzdz"] Jul 09 14:15:15.393531 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:15:15.393503 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4551179_366a_4b3a_a07e_12e4d2d32994.slice/crio-37d06ff14aeae1e5807f1853a561fb33a432b8fcfe1a53d26dae0ca077f1b895 WatchSource:0}: Error finding container 37d06ff14aeae1e5807f1853a561fb33a432b8fcfe1a53d26dae0ca077f1b895: Status 404 returned error can't find the container with id 37d06ff14aeae1e5807f1853a561fb33a432b8fcfe1a53d26dae0ca077f1b895 Jul 09 14:15:15.449658 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.449624 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mpgmz"] Jul 09 14:15:15.453477 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:15:15.453439 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a0409b2_5366_4b47_9fd8_1f893c1d43d0.slice/crio-09087ce485b3caef8e5c00e25645e49eaba9e8b3197b27a65391de885e397649 WatchSource:0}: Error finding container 09087ce485b3caef8e5c00e25645e49eaba9e8b3197b27a65391de885e397649: Status 404 returned error can't find the container with id 09087ce485b3caef8e5c00e25645e49eaba9e8b3197b27a65391de885e397649 Jul 09 14:15:15.972248 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.972181 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:15:15.975247 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.975173 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 14:15:15.975408 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:15.975252 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cs7kw\"" Jul 09 14:15:16.153893 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:16.153778 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" event={"ID":"741c6816-f1eb-478a-a1f2-f77d86cf08d3","Type":"ContainerStarted","Data":"7d7adbd48a38c1a50f45fc256cd028234051ecd8ee8d0392d267991d2fa32dd0"} Jul 09 14:15:16.153893 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:16.153842 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" event={"ID":"741c6816-f1eb-478a-a1f2-f77d86cf08d3","Type":"ContainerStarted","Data":"d92e4c886a7180be2421b08083f8751e6b38994686e942b46f76a21d09330fcb"} Jul 09 14:15:16.154265 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:16.154222 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:16.156662 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:16.156585 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9zzdz" event={"ID":"a4551179-366a-4b3a-a07e-12e4d2d32994","Type":"ContainerStarted","Data":"37d06ff14aeae1e5807f1853a561fb33a432b8fcfe1a53d26dae0ca077f1b895"} Jul 09 14:15:16.158195 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:16.158132 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mpgmz" event={"ID":"9a0409b2-5366-4b47-9fd8-1f893c1d43d0","Type":"ContainerStarted","Data":"09087ce485b3caef8e5c00e25645e49eaba9e8b3197b27a65391de885e397649"} Jul 09 14:15:16.160646 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:16.160573 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kksh7" event={"ID":"a3216d15-42b7-49e7-b5b8-01f35726265b","Type":"ContainerStarted","Data":"a138a4cd814e0b62a5858dd98318e95c7638d8e77641264fd2bbf3cf21ab8ccb"} Jul 09 14:15:16.178415 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:16.177644 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" podStartSLOduration=10.177623566 podStartE2EDuration="10.177623566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:16.176525039 +0000 UTC m=+36.299664417" watchObservedRunningTime="2026-07-09 14:15:16.177623566 +0000 UTC m=+36.300762953" Jul 09 14:15:20.983094 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:20.983060 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8mkld"] Jul 09 14:15:21.079610 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.079576 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.083976 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.083501 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 14:15:21.083976 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.083790 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 14:15:21.084198 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.084023 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 14:15:21.084198 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.084085 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 14:15:21.084198 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.084093 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 14:15:21.084198 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.084134 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 14:15:21.084768 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.084737 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fjdmz\"" Jul 09 14:15:21.230722 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.230689 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-tls\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.230885 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.230736 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-wtmp\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.230885 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.230762 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7e93b15-3033-4616-9231-edd740d1f46b-metrics-client-ca\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.230885 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.230844 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-accelerators-collector-config\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.230885 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.230871 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wghps\" (UniqueName: \"kubernetes.io/projected/e7e93b15-3033-4616-9231-edd740d1f46b-kube-api-access-wghps\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.231058 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.230923 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e7e93b15-3033-4616-9231-edd740d1f46b-root\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.231058 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.230943 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.231058 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.231017 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-textfile\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.231153 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.231059 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7e93b15-3033-4616-9231-edd740d1f46b-sys\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.332431 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.332351 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7e93b15-3033-4616-9231-edd740d1f46b-sys\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.332431 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.332390 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-tls\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.332431 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.332427 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-wtmp\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.332625 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.332479 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7e93b15-3033-4616-9231-edd740d1f46b-sys\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.332625 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.332526 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-wtmp\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.332625 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.332539 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7e93b15-3033-4616-9231-edd740d1f46b-metrics-client-ca\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.332625 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.332576 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-accelerators-collector-config\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.332625 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.332606 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wghps\" (UniqueName: \"kubernetes.io/projected/e7e93b15-3033-4616-9231-edd740d1f46b-kube-api-access-wghps\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.332893 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.332676 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e7e93b15-3033-4616-9231-edd740d1f46b-root\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.332893 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.332706 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.332893 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.332742 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-textfile\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.332893 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.332792 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e7e93b15-3033-4616-9231-edd740d1f46b-root\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.333099 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.333080 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-textfile\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.333222 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.333202 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7e93b15-3033-4616-9231-edd740d1f46b-metrics-client-ca\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.333291 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.333219 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-accelerators-collector-config\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.336220 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.336191 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.336330 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.336268 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e7e93b15-3033-4616-9231-edd740d1f46b-node-exporter-tls\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.343381 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.343360 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wghps\" (UniqueName: \"kubernetes.io/projected/e7e93b15-3033-4616-9231-edd740d1f46b-kube-api-access-wghps\") pod \"node-exporter-8mkld\" (UID: \"e7e93b15-3033-4616-9231-edd740d1f46b\") " pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.392591 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:21.392555 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8mkld" Jul 09 14:15:21.780702 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:15:21.780647 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7e93b15_3033_4616_9231_edd740d1f46b.slice/crio-a85ac3b783136e23cc6ece6c486ea15078295979ed891380417f8e571f872aca WatchSource:0}: Error finding container a85ac3b783136e23cc6ece6c486ea15078295979ed891380417f8e571f872aca: Status 404 returned error can't find the container with id a85ac3b783136e23cc6ece6c486ea15078295979ed891380417f8e571f872aca Jul 09 14:15:22.043738 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.043666 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:15:22.061281 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.061252 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:15:22.061425 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.061406 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.064510 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.064487 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 14:15:22.064639 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.064507 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 14:15:22.064639 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.064525 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 14:15:22.065126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.064897 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 14:15:22.065126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.064920 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 14:15:22.065126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.064937 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 14:15:22.065126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.064999 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 14:15:22.065126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.064938 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 14:15:22.065126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.065000 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dx5nj\"" Jul 09 14:15:22.065126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.065006 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 14:15:22.137426 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.137390 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.137426 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.137430 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/444dcf65-3841-445f-ac9e-6f69b1c35fa1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.137627 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.137448 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.137627 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.137519 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.137627 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.137565 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/444dcf65-3841-445f-ac9e-6f69b1c35fa1-config-out\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.137627 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.137592 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj4xw\" (UniqueName: \"kubernetes.io/projected/444dcf65-3841-445f-ac9e-6f69b1c35fa1-kube-api-access-gj4xw\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.137758 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.137637 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.137758 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.137686 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.137758 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.137735 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-web-config\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.137856 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.137798 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.137856 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.137834 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.137923 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.137865 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.137923 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.137880 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-config-volume\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.183279 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.183239 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k6p6r" event={"ID":"ace2890b-11e9-4efe-8a64-5fc7584e7254","Type":"ContainerStarted","Data":"d656be08fb50d183bbacee9d57633d33f3b8eb51602a1a5dcb56d3fa938f9941"} Jul 09 14:15:22.184908 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.184880 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kksh7" event={"ID":"a3216d15-42b7-49e7-b5b8-01f35726265b","Type":"ContainerStarted","Data":"676fc627b4b17260295bda13db5696c0857b90d31629e6155e36c0c0e50d2e03"} Jul 09 14:15:22.186218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.186187 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8mkld" event={"ID":"e7e93b15-3033-4616-9231-edd740d1f46b","Type":"ContainerStarted","Data":"a85ac3b783136e23cc6ece6c486ea15078295979ed891380417f8e571f872aca"} Jul 09 14:15:22.225645 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.225584 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kksh7" podStartSLOduration=1.799692796 podStartE2EDuration="8.225560625s" totalImagesPullingTime="6.425867829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:15:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:22.22405817 +0000 UTC m=+42.347197586" watchObservedRunningTime="2026-07-09 14:15:22.225560625 +0000 UTC m=+42.348700009" Jul 09 14:15:22.238808 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.238775 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.238969 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.238825 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.238969 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.238875 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-web-config\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.238969 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.238918 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.239128 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.238976 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.239128 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.239018 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.239128 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:22.239058 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-trusted-ca-bundle podName:444dcf65-3841-445f-ac9e-6f69b1c35fa1 nodeName:}" failed. No retries permitted until 2026-07-09 14:15:22.73903059 +0000 UTC m=+42.862169965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1") : configmap references non-existent config key: ca-bundle.crt Jul 09 14:15:22.239128 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.239089 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-config-volume\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.239335 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.239142 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.239335 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.239185 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/444dcf65-3841-445f-ac9e-6f69b1c35fa1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.239335 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.239213 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.240462 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.240430 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.242678 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.242081 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.242678 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.242135 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/444dcf65-3841-445f-ac9e-6f69b1c35fa1-config-out\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.242678 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.242168 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj4xw\" (UniqueName: \"kubernetes.io/projected/444dcf65-3841-445f-ac9e-6f69b1c35fa1-kube-api-access-gj4xw\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.242678 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.242252 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.242678 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:22.242563 2581 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 09 14:15:22.242678 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:15:22.242636 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-main-tls podName:444dcf65-3841-445f-ac9e-6f69b1c35fa1 nodeName:}" failed. No retries permitted until 2026-07-09 14:15:22.742619564 +0000 UTC m=+42.865758934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1") : secret "alertmanager-main-tls" not found Jul 09 14:15:22.244924 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.243870 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.248244 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.248218 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.248827 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.248408 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.248827 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.248528 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/444dcf65-3841-445f-ac9e-6f69b1c35fa1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.248827 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.248643 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.248827 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.248748 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-web-config\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.249237 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.249206 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-config-volume\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.249853 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.249833 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/444dcf65-3841-445f-ac9e-6f69b1c35fa1-config-out\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.252029 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.252007 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj4xw\" (UniqueName: \"kubernetes.io/projected/444dcf65-3841-445f-ac9e-6f69b1c35fa1-kube-api-access-gj4xw\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.746906 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.746879 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.747058 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.746924 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.747723 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.747696 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.749264 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.749235 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:22.985363 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:22.985287 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:15:23.156643 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:23.156620 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:15:23.157437 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:15:23.157414 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod444dcf65_3841_445f_ac9e_6f69b1c35fa1.slice/crio-0fd1d03df825a545f2960d6db41f6e6aee10c6d253840f9dfc53e4633ba6efeb WatchSource:0}: Error finding container 0fd1d03df825a545f2960d6db41f6e6aee10c6d253840f9dfc53e4633ba6efeb: Status 404 returned error can't find the container with id 0fd1d03df825a545f2960d6db41f6e6aee10c6d253840f9dfc53e4633ba6efeb Jul 09 14:15:23.191739 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:23.191705 2581 generic.go:356] "Generic (PLEG): container finished" podID="ace2890b-11e9-4efe-8a64-5fc7584e7254" containerID="d656be08fb50d183bbacee9d57633d33f3b8eb51602a1a5dcb56d3fa938f9941" exitCode=0 Jul 09 14:15:23.191865 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:23.191784 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k6p6r" event={"ID":"ace2890b-11e9-4efe-8a64-5fc7584e7254","Type":"ContainerDied","Data":"d656be08fb50d183bbacee9d57633d33f3b8eb51602a1a5dcb56d3fa938f9941"} Jul 09 14:15:23.193501 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:23.193475 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9zzdz" event={"ID":"a4551179-366a-4b3a-a07e-12e4d2d32994","Type":"ContainerStarted","Data":"ea8408b3a4d7a561fe2346f797046832584b079d4ee181f72c7d4491bfd34a88"} Jul 09 14:15:23.195180 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:23.195149 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mpgmz" event={"ID":"9a0409b2-5366-4b47-9fd8-1f893c1d43d0","Type":"ContainerStarted","Data":"af3959addd662f7d8f53bd8912cc46e6010f5ab509f8b71d33afdd3b09f68573"} Jul 09 14:15:23.196473 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:23.196424 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerStarted","Data":"0fd1d03df825a545f2960d6db41f6e6aee10c6d253840f9dfc53e4633ba6efeb"} Jul 09 14:15:23.976606 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:23.976558 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k"] Jul 09 14:15:24.003469 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.003433 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k"] Jul 09 14:15:24.003622 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.003574 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.006571 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.006548 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 09 14:15:24.013262 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.013214 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d1vi8ebhuf1uj\"" Jul 09 14:15:24.013262 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.013255 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-srbqc\"" Jul 09 14:15:24.013441 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.013285 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 09 14:15:24.013441 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.013246 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 09 14:15:24.013441 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.013286 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 09 14:15:24.013441 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.013246 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 09 14:15:24.159044 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.159003 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.159483 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.159132 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-grpc-tls\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.159483 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.159171 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrx4s\" (UniqueName: \"kubernetes.io/projected/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-kube-api-access-rrx4s\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.159483 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.159274 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-tls\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.159483 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.159313 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.159483 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.159381 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-metrics-client-ca\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.159483 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.159417 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.159483 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.159460 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.203380 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.203346 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9zzdz" event={"ID":"a4551179-366a-4b3a-a07e-12e4d2d32994","Type":"ContainerStarted","Data":"0d5b6d6f6a31caffbfc1c7eb4f8f6fab1929aeffacd75595961dd746faea6a01"} Jul 09 14:15:24.205420 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.205379 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mpgmz" event={"ID":"9a0409b2-5366-4b47-9fd8-1f893c1d43d0","Type":"ContainerStarted","Data":"bf6f2a04b419b12a992d66cfbb62788680eebc4d5b6c2058a21a351f99024761"} Jul 09 14:15:24.205573 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.205536 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:24.209017 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.208930 2581 generic.go:356] "Generic (PLEG): container finished" podID="ace2890b-11e9-4efe-8a64-5fc7584e7254" containerID="4feebaa722421c7d00d3b08b880eb737a0011837d57534427eb39929af333d3a" exitCode=0 Jul 09 14:15:24.209017 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.208986 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k6p6r" event={"ID":"ace2890b-11e9-4efe-8a64-5fc7584e7254","Type":"ContainerDied","Data":"4feebaa722421c7d00d3b08b880eb737a0011837d57534427eb39929af333d3a"} Jul 09 14:15:24.222686 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.222640 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mpgmz" podStartSLOduration=2.655449961 podStartE2EDuration="10.222621006s" totalImagesPullingTime="7.567171045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:15:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:24.221473693 +0000 UTC m=+44.344613078" watchObservedRunningTime="2026-07-09 14:15:24.222621006 +0000 UTC m=+44.345760390" Jul 09 14:15:24.261023 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.260986 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.261191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.261084 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.261191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.261144 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-grpc-tls\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.261191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.261176 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrx4s\" (UniqueName: \"kubernetes.io/projected/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-kube-api-access-rrx4s\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.261349 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.261224 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-tls\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.261349 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.261260 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.261349 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.261307 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-metrics-client-ca\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.261349 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.261337 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.265026 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.263126 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-metrics-client-ca\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.265026 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.264270 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.265026 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.264674 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.266163 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.265120 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.266163 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.265774 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-grpc-tls\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.266163 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.266047 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-tls\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.266935 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.266908 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.274619 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.274594 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrx4s\" (UniqueName: \"kubernetes.io/projected/cd9d70aa-c3ff-4be6-9508-e73c33a43e98-kube-api-access-rrx4s\") pod \"thanos-querier-d5dccd9cc-5tb7k\" (UID: \"cd9d70aa-c3ff-4be6-9508-e73c33a43e98\") " pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.312213 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.312184 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:24.487345 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:24.487311 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k"] Jul 09 14:15:25.075569 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:15:25.075526 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd9d70aa_c3ff_4be6_9508_e73c33a43e98.slice/crio-ec291081a7a765eaf636a02f0320e845ef661e81ed2b3b2ae3ec664373eb7a7e WatchSource:0}: Error finding container ec291081a7a765eaf636a02f0320e845ef661e81ed2b3b2ae3ec664373eb7a7e: Status 404 returned error can't find the container with id ec291081a7a765eaf636a02f0320e845ef661e81ed2b3b2ae3ec664373eb7a7e Jul 09 14:15:25.216743 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.216696 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k6p6r" event={"ID":"ace2890b-11e9-4efe-8a64-5fc7584e7254","Type":"ContainerStarted","Data":"bf40fe4d0f963d755ff848dc79a80594c0f719718746c38430bb2b27c98d0c3c"} Jul 09 14:15:25.218171 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.218142 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" event={"ID":"cd9d70aa-c3ff-4be6-9508-e73c33a43e98","Type":"ContainerStarted","Data":"ec291081a7a765eaf636a02f0320e845ef661e81ed2b3b2ae3ec664373eb7a7e"} Jul 09 14:15:25.249723 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.249689 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6f87466b5b-72vqb"] Jul 09 14:15:25.269488 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.269454 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.277890 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.277643 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 09 14:15:25.277890 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.277674 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 09 14:15:25.278507 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.277906 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-ntf62\"" Jul 09 14:15:25.278507 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.277967 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fl01pdl3njfe\"" Jul 09 14:15:25.278507 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.278007 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 09 14:15:25.284991 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.284931 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-k6p6r" podStartSLOduration=8.812031784 podStartE2EDuration="45.284918025s" totalImagesPullingTime="36.472886241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:14:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:25.264371077 +0000 UTC m=+45.387510480" watchObservedRunningTime="2026-07-09 14:15:25.284918025 +0000 UTC m=+45.408057407" Jul 09 14:15:25.289134 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.289113 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 14:15:25.291597 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.291573 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f87466b5b-72vqb"] Jul 09 14:15:25.373565 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.373530 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ae7cac9-ee24-4331-aa0e-796fecabe79d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.374032 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.373966 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2ae7cac9-ee24-4331-aa0e-796fecabe79d-metrics-server-audit-profiles\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.374149 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.374071 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2ae7cac9-ee24-4331-aa0e-796fecabe79d-secret-metrics-server-client-certs\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.374149 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.374123 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2ae7cac9-ee24-4331-aa0e-796fecabe79d-secret-metrics-server-tls\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.374309 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.374287 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ae7cac9-ee24-4331-aa0e-796fecabe79d-client-ca-bundle\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.374393 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.374350 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2ae7cac9-ee24-4331-aa0e-796fecabe79d-audit-log\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.374393 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.374386 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c86v8\" (UniqueName: \"kubernetes.io/projected/2ae7cac9-ee24-4331-aa0e-796fecabe79d-kube-api-access-c86v8\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.475637 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.475604 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2ae7cac9-ee24-4331-aa0e-796fecabe79d-metrics-server-audit-profiles\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.475805 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.475641 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2ae7cac9-ee24-4331-aa0e-796fecabe79d-secret-metrics-server-client-certs\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.475805 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.475679 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2ae7cac9-ee24-4331-aa0e-796fecabe79d-secret-metrics-server-tls\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.475805 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.475735 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ae7cac9-ee24-4331-aa0e-796fecabe79d-client-ca-bundle\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.475805 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.475768 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2ae7cac9-ee24-4331-aa0e-796fecabe79d-audit-log\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.475805 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.475792 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c86v8\" (UniqueName: \"kubernetes.io/projected/2ae7cac9-ee24-4331-aa0e-796fecabe79d-kube-api-access-c86v8\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.476081 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.475819 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ae7cac9-ee24-4331-aa0e-796fecabe79d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.476565 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.476435 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2ae7cac9-ee24-4331-aa0e-796fecabe79d-audit-log\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.476565 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.476478 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ae7cac9-ee24-4331-aa0e-796fecabe79d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.476882 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.476852 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2ae7cac9-ee24-4331-aa0e-796fecabe79d-metrics-server-audit-profiles\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.478785 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.478759 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2ae7cac9-ee24-4331-aa0e-796fecabe79d-secret-metrics-server-client-certs\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.478991 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.478968 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ae7cac9-ee24-4331-aa0e-796fecabe79d-client-ca-bundle\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.479184 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.479163 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2ae7cac9-ee24-4331-aa0e-796fecabe79d-secret-metrics-server-tls\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.485692 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.485643 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c86v8\" (UniqueName: \"kubernetes.io/projected/2ae7cac9-ee24-4331-aa0e-796fecabe79d-kube-api-access-c86v8\") pod \"metrics-server-6f87466b5b-72vqb\" (UID: \"2ae7cac9-ee24-4331-aa0e-796fecabe79d\") " pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:25.580834 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:25.580796 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:26.147389 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.147355 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7df788b5dd-qvh48"] Jul 09 14:15:26.213641 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.213594 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7df788b5dd-qvh48"] Jul 09 14:15:26.213834 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.213735 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.216688 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.216651 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 09 14:15:26.216986 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.216939 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 09 14:15:26.217388 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.217115 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 09 14:15:26.217388 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.217228 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 09 14:15:26.217388 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.217288 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-rw5d6\"" Jul 09 14:15:26.217388 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.217329 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 09 14:15:26.222749 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.222718 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 09 14:15:26.224562 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.224512 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8mkld" event={"ID":"e7e93b15-3033-4616-9231-edd740d1f46b","Type":"ContainerStarted","Data":"9dc5bc19fdb5221831a0a08c2b82db86abdbfbf3a00ffc726bac7de854087234"} Jul 09 14:15:26.226544 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.226519 2581 generic.go:356] "Generic (PLEG): container finished" podID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerID="5bdaa968cf3264e038afc6acfb41364b0271547edf717c1a31e2efd727babf7d" exitCode=0 Jul 09 14:15:26.226674 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.226588 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerDied","Data":"5bdaa968cf3264e038afc6acfb41364b0271547edf717c1a31e2efd727babf7d"} Jul 09 14:15:26.384096 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.384061 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-telemeter-client-tls\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.384257 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.384147 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-metrics-client-ca\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.384257 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.384215 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-secret-telemeter-client\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.384403 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.384292 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbhjx\" (UniqueName: \"kubernetes.io/projected/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-kube-api-access-rbhjx\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.384456 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.384438 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-serving-certs-ca-bundle\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.384510 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.384473 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.384594 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.384577 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-federate-client-tls\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.384637 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.384616 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.486110 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.486024 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbhjx\" (UniqueName: \"kubernetes.io/projected/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-kube-api-access-rbhjx\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.486110 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.486101 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-serving-certs-ca-bundle\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.486340 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.486136 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.486340 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.486191 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-federate-client-tls\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.486340 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.486218 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.486340 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.486283 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-telemeter-client-tls\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.486340 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.486305 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-metrics-client-ca\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.486340 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.486342 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-secret-telemeter-client\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.487281 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.486871 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-serving-certs-ca-bundle\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.487502 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.487435 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-metrics-client-ca\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.487684 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.487660 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.489224 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.489197 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-telemeter-client-tls\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.489330 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.489297 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-federate-client-tls\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.489330 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.489319 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-secret-telemeter-client\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.489423 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.489343 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.494654 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.494631 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbhjx\" (UniqueName: \"kubernetes.io/projected/c77a677a-c1c4-477d-ae26-0b80b6ef2a2a-kube-api-access-rbhjx\") pod \"telemeter-client-7df788b5dd-qvh48\" (UID: \"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a\") " pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.526063 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.526029 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" Jul 09 14:15:26.876692 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.876660 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f87466b5b-72vqb"] Jul 09 14:15:26.880167 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:15:26.880139 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ae7cac9_ee24_4331_aa0e_796fecabe79d.slice/crio-765aecb8a086a1271b62bb4512baf3f26ba6ed53308a08e23f6a45e9d106c30a WatchSource:0}: Error finding container 765aecb8a086a1271b62bb4512baf3f26ba6ed53308a08e23f6a45e9d106c30a: Status 404 returned error can't find the container with id 765aecb8a086a1271b62bb4512baf3f26ba6ed53308a08e23f6a45e9d106c30a Jul 09 14:15:26.896386 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:26.896358 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7df788b5dd-qvh48"] Jul 09 14:15:26.899272 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:15:26.899247 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc77a677a_c1c4_477d_ae26_0b80b6ef2a2a.slice/crio-4472e6632a881533381b3d4934101557a99846aa119956023fe9b238b76b8b98 WatchSource:0}: Error finding container 4472e6632a881533381b3d4934101557a99846aa119956023fe9b238b76b8b98: Status 404 returned error can't find the container with id 4472e6632a881533381b3d4934101557a99846aa119956023fe9b238b76b8b98 Jul 09 14:15:27.232357 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:27.232320 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" event={"ID":"2ae7cac9-ee24-4331-aa0e-796fecabe79d","Type":"ContainerStarted","Data":"765aecb8a086a1271b62bb4512baf3f26ba6ed53308a08e23f6a45e9d106c30a"} Jul 09 14:15:27.234024 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:27.233990 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" event={"ID":"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a","Type":"ContainerStarted","Data":"4472e6632a881533381b3d4934101557a99846aa119956023fe9b238b76b8b98"} Jul 09 14:15:27.236816 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:27.236778 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9zzdz" event={"ID":"a4551179-366a-4b3a-a07e-12e4d2d32994","Type":"ContainerStarted","Data":"31c7fc81d7d52fa96030ada26aa5e4d7e870ddeea3538bc0d2c9cf93e4e1a92d"} Jul 09 14:15:27.238740 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:27.238711 2581 generic.go:356] "Generic (PLEG): container finished" podID="e7e93b15-3033-4616-9231-edd740d1f46b" containerID="9dc5bc19fdb5221831a0a08c2b82db86abdbfbf3a00ffc726bac7de854087234" exitCode=0 Jul 09 14:15:27.238848 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:27.238755 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8mkld" event={"ID":"e7e93b15-3033-4616-9231-edd740d1f46b","Type":"ContainerDied","Data":"9dc5bc19fdb5221831a0a08c2b82db86abdbfbf3a00ffc726bac7de854087234"} Jul 09 14:15:27.256474 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:27.256400 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9zzdz" podStartSLOduration=1.744401111 podStartE2EDuration="13.256383336s" totalImagesPullingTime="11.511982225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:15:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:27.255005693 +0000 UTC m=+47.378145075" watchObservedRunningTime="2026-07-09 14:15:27.256383336 +0000 UTC m=+47.379522719" Jul 09 14:15:28.245218 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:28.245164 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8mkld" event={"ID":"e7e93b15-3033-4616-9231-edd740d1f46b","Type":"ContainerStarted","Data":"676840ca4cb0e857ebe54ecc35515e7760860127e65a743fcc46f75550fe7245"} Jul 09 14:15:28.245650 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:28.245228 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8mkld" event={"ID":"e7e93b15-3033-4616-9231-edd740d1f46b","Type":"ContainerStarted","Data":"608b6bd6026ff080f377caa5a5a1f799e06d0caaa9916919c21d6a2728e4fd40"} Jul 09 14:15:28.268787 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:28.268725 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8mkld" podStartSLOduration=4.566647409 podStartE2EDuration="8.268704158s" totalImagesPullingTime="3.593178609s" totalInitContainerRuntime="108.87814ms" isStatefulPod=true podCreationTimestamp="2026-07-09 14:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:28.267091157 +0000 UTC m=+48.390230541" watchObservedRunningTime="2026-07-09 14:15:28.268704158 +0000 UTC m=+48.391843520" Jul 09 14:15:30.256968 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:30.256925 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerStarted","Data":"fbdfcb9d53f1e511fff41b18b66c0820bb6d41154fedc3f7b4a5cf3d90a75879"} Jul 09 14:15:30.257337 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:30.257037 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerStarted","Data":"912dab06057c1a1b893ac0b035f98427835c9139e850a926195866e2e6e5c859"} Jul 09 14:15:30.257337 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:30.257080 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerStarted","Data":"ae36e920e1c27d0c2c1b143e9e873a46e2e25550f37425eda105c384d5380176"} Jul 09 14:15:30.257337 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:30.257117 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerStarted","Data":"8c80ba50c05e3e823a7d77b8117288e29560e6315a2238fb88faa7271a62d18d"} Jul 09 14:15:30.258979 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:30.258843 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" event={"ID":"2ae7cac9-ee24-4331-aa0e-796fecabe79d","Type":"ContainerStarted","Data":"0e32e9d569bc0e81ec69196445e24fced567259ab79945d0a362fbef0bd4e3d1"} Jul 09 14:15:30.262074 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:30.261917 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" event={"ID":"cd9d70aa-c3ff-4be6-9508-e73c33a43e98","Type":"ContainerStarted","Data":"f35c77e768fb45005a8ad81123b76f0b7bc821a9d5aefb6a305329f14e71c029"} Jul 09 14:15:30.262074 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:30.262009 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" event={"ID":"cd9d70aa-c3ff-4be6-9508-e73c33a43e98","Type":"ContainerStarted","Data":"ef6c3bdabb6fb0e975f9a29a7c9bdaaabb32da0dceddffcf73c24a9cc5f32021"} Jul 09 14:15:30.262074 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:30.262052 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" event={"ID":"cd9d70aa-c3ff-4be6-9508-e73c33a43e98","Type":"ContainerStarted","Data":"25a64ade555b0250c2c77de76e7256dd87b3eb09cbd3146bb799ffff27210c8a"} Jul 09 14:15:30.281092 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:30.280939 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" podStartSLOduration=2.343990282 podStartE2EDuration="5.28092187s" totalImagesPullingTime="2.936931588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:15:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:30.279878572 +0000 UTC m=+50.403017959" watchObservedRunningTime="2026-07-09 14:15:30.28092187 +0000 UTC m=+50.404061253" Jul 09 14:15:31.269187 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:31.269140 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerStarted","Data":"478280f388173af8f995287de6eccb35c078ae2c0b09df268cf71c5353b5e3bf"} Jul 09 14:15:32.278478 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:32.278442 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" event={"ID":"cd9d70aa-c3ff-4be6-9508-e73c33a43e98","Type":"ContainerStarted","Data":"c4bf38cd6a9478e2dcc7e80b4a372f4b84e03ce944802bf8ac9aeba153ef4318"} Jul 09 14:15:32.278810 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:32.278530 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" event={"ID":"cd9d70aa-c3ff-4be6-9508-e73c33a43e98","Type":"ContainerStarted","Data":"9c6d59c4396d7c175c9090985e471e643763461ca14a3dc2ca9f76cf963224fa"} Jul 09 14:15:33.100093 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.100063 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77d69d764d-fn55z"] Jul 09 14:15:33.119447 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.119410 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77d69d764d-fn55z"] Jul 09 14:15:33.119569 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.119528 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.122535 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.122511 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 14:15:33.122669 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.122511 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 14:15:33.122669 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.122517 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mb22p\"" Jul 09 14:15:33.122669 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.122617 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 14:15:33.123943 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.123921 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 14:15:33.124101 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.124033 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 14:15:33.124314 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.124296 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 14:15:33.124417 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.124339 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 14:15:33.131974 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.129360 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 14:15:33.253592 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.253553 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-console-config\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.253732 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.253601 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-service-ca\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.253732 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.253647 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e4836bb-d56f-437a-893f-83160d58f2d0-console-serving-cert\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.253732 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.253675 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e4836bb-d56f-437a-893f-83160d58f2d0-console-oauth-config\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.253732 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.253689 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-oauth-serving-cert\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.253879 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.253788 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-797f9\" (UniqueName: \"kubernetes.io/projected/6e4836bb-d56f-437a-893f-83160d58f2d0-kube-api-access-797f9\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.253879 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.253821 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-trusted-ca-bundle\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.283808 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.283772 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" event={"ID":"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a","Type":"ContainerStarted","Data":"4e45149eefcaffdc65c00f80efca0307293e90e15777dfb1ede57ed625d258be"} Jul 09 14:15:33.284240 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.283884 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" event={"ID":"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a","Type":"ContainerStarted","Data":"4268c2e15cfdef86d02de3ebc142a4cd3af28c6fbb164d18cfe2409493d3a34f"} Jul 09 14:15:33.284240 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.283918 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" event={"ID":"c77a677a-c1c4-477d-ae26-0b80b6ef2a2a","Type":"ContainerStarted","Data":"e8abf436456114b24ee2f78a3af7c7abeebe2b04ff6dfb3cb84242654ec62aec"} Jul 09 14:15:33.286897 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.286872 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" event={"ID":"cd9d70aa-c3ff-4be6-9508-e73c33a43e98","Type":"ContainerStarted","Data":"6f811059a3706feaf0fd89b86580d773107e6888abd7291674a3f07d5211599c"} Jul 09 14:15:33.287037 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.286974 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:33.290355 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.290332 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerStarted","Data":"a66f59b512bb401420d7d3398d8af76c960b052ef282b96b42b3fc77f561756d"} Jul 09 14:15:33.306125 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.306081 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7df788b5dd-qvh48" podStartSLOduration=1.70422338 podStartE2EDuration="7.306068682s" totalImagesPullingTime="5.601845302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:33.304407102 +0000 UTC m=+53.427546530" watchObservedRunningTime="2026-07-09 14:15:33.306068682 +0000 UTC m=+53.429208043" Jul 09 14:15:33.336269 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.336221 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.082291614 podStartE2EDuration="11.336206198s" totalImagesPullingTime="8.245088102s" totalInitContainerRuntime="8.826482ms" isStatefulPod=false podCreationTimestamp="2026-07-09 14:15:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:33.334405827 +0000 UTC m=+53.457545235" watchObservedRunningTime="2026-07-09 14:15:33.336206198 +0000 UTC m=+53.459345581" Jul 09 14:15:33.354567 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.354492 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e4836bb-d56f-437a-893f-83160d58f2d0-console-oauth-config\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.354567 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.354524 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-oauth-serving-cert\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.354567 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.354565 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-797f9\" (UniqueName: \"kubernetes.io/projected/6e4836bb-d56f-437a-893f-83160d58f2d0-kube-api-access-797f9\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.354810 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.354592 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-trusted-ca-bundle\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.354810 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.354637 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-console-config\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.354810 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.354677 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-service-ca\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.354810 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.354727 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e4836bb-d56f-437a-893f-83160d58f2d0-console-serving-cert\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.355367 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.355344 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-oauth-serving-cert\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.355446 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.355360 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-console-config\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.355654 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.355633 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-service-ca\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.355725 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.355702 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-trusted-ca-bundle\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.357626 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.357598 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e4836bb-d56f-437a-893f-83160d58f2d0-console-serving-cert\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.357726 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.357712 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e4836bb-d56f-437a-893f-83160d58f2d0-console-oauth-config\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.362669 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.362619 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" podStartSLOduration=4.485989971 podStartE2EDuration="10.362601348s" totalImagesPullingTime="5.876611377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:15:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:33.36103498 +0000 UTC m=+53.484174364" watchObservedRunningTime="2026-07-09 14:15:33.362601348 +0000 UTC m=+53.485740733" Jul 09 14:15:33.365671 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.365644 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-797f9\" (UniqueName: \"kubernetes.io/projected/6e4836bb-d56f-437a-893f-83160d58f2d0-kube-api-access-797f9\") pod \"console-77d69d764d-fn55z\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.429354 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.429320 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:33.581648 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:33.581621 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77d69d764d-fn55z"] Jul 09 14:15:33.584210 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:15:33.584176 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e4836bb_d56f_437a_893f_83160d58f2d0.slice/crio-85af2d3b0648e1f8f3f3e9206c6c1beda68734df46c57f6ed3408ce2b41db3be WatchSource:0}: Error finding container 85af2d3b0648e1f8f3f3e9206c6c1beda68734df46c57f6ed3408ce2b41db3be: Status 404 returned error can't find the container with id 85af2d3b0648e1f8f3f3e9206c6c1beda68734df46c57f6ed3408ce2b41db3be Jul 09 14:15:34.220918 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:34.220887 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mpgmz" Jul 09 14:15:34.295598 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:34.295556 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77d69d764d-fn55z" event={"ID":"6e4836bb-d56f-437a-893f-83160d58f2d0","Type":"ContainerStarted","Data":"85af2d3b0648e1f8f3f3e9206c6c1beda68734df46c57f6ed3408ce2b41db3be"} Jul 09 14:15:37.169101 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:37.169073 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-9f8cc8f45-kvxn7" Jul 09 14:15:39.301874 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:39.301845 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-d5dccd9cc-5tb7k" Jul 09 14:15:40.322823 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.322782 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77d69d764d-fn55z" event={"ID":"6e4836bb-d56f-437a-893f-83160d58f2d0","Type":"ContainerStarted","Data":"57330ceffd0238800ef37883362cbbe0ef04b47aeffe86627b9ee257006ed556"} Jul 09 14:15:40.344482 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.344423 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77d69d764d-fn55z" podStartSLOduration=1.427065411 podStartE2EDuration="7.344407372s" totalImagesPullingTime="5.917341961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:40.34327907 +0000 UTC m=+60.466418453" watchObservedRunningTime="2026-07-09 14:15:40.344407372 +0000 UTC m=+60.467546755" Jul 09 14:15:40.749178 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.749142 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77d69d764d-fn55z"] Jul 09 14:15:40.780523 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.780493 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cb99b5996-b4nh6"] Jul 09 14:15:40.783150 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.783134 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.794855 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.794830 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb99b5996-b4nh6"] Jul 09 14:15:40.826129 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.826103 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-oauth-config\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.826262 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.826168 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-trusted-ca-bundle\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.826262 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.826192 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-service-ca\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.826262 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.826227 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wzkg\" (UniqueName: \"kubernetes.io/projected/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-kube-api-access-5wzkg\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.826385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.826314 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-config\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.826385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.826344 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-oauth-serving-cert\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.826385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.826376 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-serving-cert\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.927307 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.927267 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-trusted-ca-bundle\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.927307 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.927307 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-service-ca\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.927563 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.927325 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wzkg\" (UniqueName: \"kubernetes.io/projected/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-kube-api-access-5wzkg\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.927563 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.927474 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-config\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.927563 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.927514 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-oauth-serving-cert\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.927563 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.927562 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-serving-cert\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.927779 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.927717 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-oauth-config\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.928113 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.928081 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-service-ca\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.928246 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.928148 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-config\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.928246 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.928202 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-trusted-ca-bundle\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.928246 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.928216 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-oauth-serving-cert\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.930178 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.930148 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-oauth-config\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.930277 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.930156 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-serving-cert\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:40.936241 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:40.936215 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wzkg\" (UniqueName: \"kubernetes.io/projected/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-kube-api-access-5wzkg\") pod \"console-5cb99b5996-b4nh6\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:41.091915 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:41.091881 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:41.150923 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:41.150889 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v67ml" Jul 09 14:15:41.235031 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:41.235007 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb99b5996-b4nh6"] Jul 09 14:15:41.237482 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:15:41.237446 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8589e22a_3e4a_4b1a_9427_7ed75f6514dc.slice/crio-9de3a23db0e66499b434e187c66b9d0ff6e646d0a98fae538397eedfdf870172 WatchSource:0}: Error finding container 9de3a23db0e66499b434e187c66b9d0ff6e646d0a98fae538397eedfdf870172: Status 404 returned error can't find the container with id 9de3a23db0e66499b434e187c66b9d0ff6e646d0a98fae538397eedfdf870172 Jul 09 14:15:41.328940 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:41.328903 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb99b5996-b4nh6" event={"ID":"8589e22a-3e4a-4b1a-9427-7ed75f6514dc","Type":"ContainerStarted","Data":"109cb26ad2306de16d05a9314a4a5bf64acbd572cc817e282d6652acf099cb6f"} Jul 09 14:15:41.329421 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:41.328993 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb99b5996-b4nh6" event={"ID":"8589e22a-3e4a-4b1a-9427-7ed75f6514dc","Type":"ContainerStarted","Data":"9de3a23db0e66499b434e187c66b9d0ff6e646d0a98fae538397eedfdf870172"} Jul 09 14:15:41.352550 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:41.352431 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cb99b5996-b4nh6" podStartSLOduration=1.3524129 podStartE2EDuration="1.3524129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:41.351017438 +0000 UTC m=+61.474156841" watchObservedRunningTime="2026-07-09 14:15:41.3524129 +0000 UTC m=+61.475552284" Jul 09 14:15:43.429999 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:43.429964 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:15:45.580988 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:45.580915 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:45.580988 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:45.580997 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:15:46.277426 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.277387 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:15:46.277627 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.277474 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs\") pod \"network-metrics-daemon-qr5pr\" (UID: \"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8\") " pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:15:46.280787 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.280767 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 14:15:46.281906 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.281891 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 14:15:46.290714 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.290693 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f9a1846-c394-4931-a7f5-d839e7558c70-original-pull-secret\") pod \"global-pull-secret-syncer-5kc7s\" (UID: \"6f9a1846-c394-4931-a7f5-d839e7558c70\") " pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:15:46.290828 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.290722 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f5c80d5-b98a-41b3-b68a-94e8f1b303d8-metrics-certs\") pod \"network-metrics-daemon-qr5pr\" (UID: \"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8\") " pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:15:46.378407 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.378370 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zdlh\" (UniqueName: \"kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh\") pod \"network-check-target-qqzlh\" (UID: \"66695e98-eb01-4479-bed6-81320c8b3586\") " pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:15:46.381372 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.381355 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 14:15:46.391944 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.391924 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 14:15:46.402601 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.402578 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zdlh\" (UniqueName: \"kubernetes.io/projected/66695e98-eb01-4479-bed6-81320c8b3586-kube-api-access-4zdlh\") pod \"network-check-target-qqzlh\" (UID: \"66695e98-eb01-4479-bed6-81320c8b3586\") " pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:15:46.487052 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.487022 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wzd4v\"" Jul 09 14:15:46.490439 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.490426 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5kc7s" Jul 09 14:15:46.494545 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.494527 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:15:46.589592 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.589563 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cs7kw\"" Jul 09 14:15:46.596589 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.596567 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qr5pr" Jul 09 14:15:46.641244 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.641213 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qqzlh"] Jul 09 14:15:46.644276 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:15:46.644247 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66695e98_eb01_4479_bed6_81320c8b3586.slice/crio-4e9b45ed890c2cad1acda49e165e8598f283973ed753c147e731539b2bfa2aaa WatchSource:0}: Error finding container 4e9b45ed890c2cad1acda49e165e8598f283973ed753c147e731539b2bfa2aaa: Status 404 returned error can't find the container with id 4e9b45ed890c2cad1acda49e165e8598f283973ed753c147e731539b2bfa2aaa Jul 09 14:15:46.659337 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.659284 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5kc7s"] Jul 09 14:15:46.663309 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:15:46.663281 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9a1846_c394_4931_a7f5_d839e7558c70.slice/crio-07bb3da540024504cdc87959c3177586062fbdfe88df290d8ca0f189e6671be2 WatchSource:0}: Error finding container 07bb3da540024504cdc87959c3177586062fbdfe88df290d8ca0f189e6671be2: Status 404 returned error can't find the container with id 07bb3da540024504cdc87959c3177586062fbdfe88df290d8ca0f189e6671be2 Jul 09 14:15:46.741123 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:46.741093 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qr5pr"] Jul 09 14:15:46.744178 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:15:46.744136 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f5c80d5_b98a_41b3_b68a_94e8f1b303d8.slice/crio-6564a3f44f86991ffa80bcae9e33ca57b05df91e129b529181d8bdcc68deda50 WatchSource:0}: Error finding container 6564a3f44f86991ffa80bcae9e33ca57b05df91e129b529181d8bdcc68deda50: Status 404 returned error can't find the container with id 6564a3f44f86991ffa80bcae9e33ca57b05df91e129b529181d8bdcc68deda50 Jul 09 14:15:47.353017 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:47.352975 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5kc7s" event={"ID":"6f9a1846-c394-4931-a7f5-d839e7558c70","Type":"ContainerStarted","Data":"07bb3da540024504cdc87959c3177586062fbdfe88df290d8ca0f189e6671be2"} Jul 09 14:15:47.354455 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:47.354407 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qr5pr" event={"ID":"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8","Type":"ContainerStarted","Data":"6564a3f44f86991ffa80bcae9e33ca57b05df91e129b529181d8bdcc68deda50"} Jul 09 14:15:47.355826 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:47.355789 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qqzlh" event={"ID":"66695e98-eb01-4479-bed6-81320c8b3586","Type":"ContainerStarted","Data":"4e9b45ed890c2cad1acda49e165e8598f283973ed753c147e731539b2bfa2aaa"} Jul 09 14:15:48.363152 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:48.363108 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qr5pr" event={"ID":"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8","Type":"ContainerStarted","Data":"10c074321e2423b4de6449bcf4ee0678a9e103c678113f24d16967bbfc2240f4"} Jul 09 14:15:48.363579 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:48.363178 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qr5pr" event={"ID":"4f5c80d5-b98a-41b3-b68a-94e8f1b303d8","Type":"ContainerStarted","Data":"df5676867fcc6c126358f4e82dc135d5e474e04cf76f698ef56df014ae33bc4b"} Jul 09 14:15:48.383798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:48.383741 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qr5pr" podStartSLOduration=67.123551614 podStartE2EDuration="1m8.383721198s" totalImagesPullingTime="1.260169584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:14:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:48.381718476 +0000 UTC m=+68.504857859" watchObservedRunningTime="2026-07-09 14:15:48.383721198 +0000 UTC m=+68.506860582" Jul 09 14:15:51.092150 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:51.092098 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:51.092603 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:51.092176 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:51.098576 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:51.098552 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:51.381897 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:51.381824 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:15:52.383024 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:52.382985 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qqzlh" event={"ID":"66695e98-eb01-4479-bed6-81320c8b3586","Type":"ContainerStarted","Data":"2271a8e0b7fbf2832e45743b571afbf17419783cea5809847b413359a7faefcc"} Jul 09 14:15:52.383505 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:52.383192 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:15:52.384699 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:52.384670 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5kc7s" event={"ID":"6f9a1846-c394-4931-a7f5-d839e7558c70","Type":"ContainerStarted","Data":"e8495e73040c6575a1ea9c9e31b864212e49adba2957bc4ec3643239fd144314"} Jul 09 14:15:52.403636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:52.403591 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qqzlh" podStartSLOduration=67.182840643 podStartE2EDuration="1m12.403575659s" totalImagesPullingTime="5.220735016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:14:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:52.403051708 +0000 UTC m=+72.526191092" watchObservedRunningTime="2026-07-09 14:15:52.403575659 +0000 UTC m=+72.526715042" Jul 09 14:15:52.419816 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:15:52.419771 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5kc7s" podStartSLOduration=67.206199233 podStartE2EDuration="1m12.419756085s" totalImagesPullingTime="5.213556852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:14:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:15:52.41893937 +0000 UTC m=+72.542078772" watchObservedRunningTime="2026-07-09 14:15:52.419756085 +0000 UTC m=+72.542895467" Jul 09 14:16:05.586896 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:05.586788 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:16:05.590592 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:05.590574 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6f87466b5b-72vqb" Jul 09 14:16:06.634381 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.634346 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6ff87f966f-qmfd5"] Jul 09 14:16:06.637182 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.637165 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.653750 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.653725 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6ff87f966f-qmfd5"] Jul 09 14:16:06.759724 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.759686 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/036b5417-59df-45e9-8f35-3f1d729b4281-console-oauth-config\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.759907 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.759731 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/036b5417-59df-45e9-8f35-3f1d729b4281-console-serving-cert\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.759907 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.759757 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-service-ca\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.759907 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.759842 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-oauth-serving-cert\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.759907 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.759889 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv8h9\" (UniqueName: \"kubernetes.io/projected/036b5417-59df-45e9-8f35-3f1d729b4281-kube-api-access-fv8h9\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.760074 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.759912 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-trusted-ca-bundle\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.760074 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.759969 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-console-config\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.861084 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.861047 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-oauth-serving-cert\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.861231 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.861094 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv8h9\" (UniqueName: \"kubernetes.io/projected/036b5417-59df-45e9-8f35-3f1d729b4281-kube-api-access-fv8h9\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.861231 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.861114 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-trusted-ca-bundle\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.861231 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.861159 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-console-config\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.861231 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.861227 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/036b5417-59df-45e9-8f35-3f1d729b4281-console-oauth-config\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.861387 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.861255 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/036b5417-59df-45e9-8f35-3f1d729b4281-console-serving-cert\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.861387 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.861287 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-service-ca\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.861827 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.861805 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-oauth-serving-cert\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.861926 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.861898 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-console-config\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.862086 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.862065 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-service-ca\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.862086 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.862074 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-trusted-ca-bundle\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.863802 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.863779 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/036b5417-59df-45e9-8f35-3f1d729b4281-console-serving-cert\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.863920 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.863897 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/036b5417-59df-45e9-8f35-3f1d729b4281-console-oauth-config\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.873105 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.873078 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv8h9\" (UniqueName: \"kubernetes.io/projected/036b5417-59df-45e9-8f35-3f1d729b4281-kube-api-access-fv8h9\") pod \"console-6ff87f966f-qmfd5\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:06.945987 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:06.945897 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:07.086500 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.086452 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6ff87f966f-qmfd5"] Jul 09 14:16:07.089710 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:16:07.089681 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod036b5417_59df_45e9_8f35_3f1d729b4281.slice/crio-a6e26afcffd0f58a1ccff6a81011cf140e2db3111c748e7ba34f2995b615e0d6 WatchSource:0}: Error finding container a6e26afcffd0f58a1ccff6a81011cf140e2db3111c748e7ba34f2995b615e0d6: Status 404 returned error can't find the container with id a6e26afcffd0f58a1ccff6a81011cf140e2db3111c748e7ba34f2995b615e0d6 Jul 09 14:16:07.352009 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.351929 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77d69d764d-fn55z" podUID="6e4836bb-d56f-437a-893f-83160d58f2d0" containerName="console" containerID="cri-o://57330ceffd0238800ef37883362cbbe0ef04b47aeffe86627b9ee257006ed556" gracePeriod=15 Jul 09 14:16:07.447388 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.447355 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff87f966f-qmfd5" event={"ID":"036b5417-59df-45e9-8f35-3f1d729b4281","Type":"ContainerStarted","Data":"caa38c67caff92d65ba0dd559205054eb3fbd8f0aedba0c9f7c9f1ab5da2ead8"} Jul 09 14:16:07.447566 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.447413 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff87f966f-qmfd5" event={"ID":"036b5417-59df-45e9-8f35-3f1d729b4281","Type":"ContainerStarted","Data":"a6e26afcffd0f58a1ccff6a81011cf140e2db3111c748e7ba34f2995b615e0d6"} Jul 09 14:16:07.466551 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.466496 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6ff87f966f-qmfd5" podStartSLOduration=1.4664779700000001 podStartE2EDuration="1.46647797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:16:07.46606085 +0000 UTC m=+87.589200234" watchObservedRunningTime="2026-07-09 14:16:07.46647797 +0000 UTC m=+87.589617352" Jul 09 14:16:07.584027 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.583999 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:16:07.769054 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.768931 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-trusted-ca-bundle\") pod \"6e4836bb-d56f-437a-893f-83160d58f2d0\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " Jul 09 14:16:07.769054 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.769028 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-797f9\" (UniqueName: \"kubernetes.io/projected/6e4836bb-d56f-437a-893f-83160d58f2d0-kube-api-access-797f9\") pod \"6e4836bb-d56f-437a-893f-83160d58f2d0\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " Jul 09 14:16:07.769571 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.769125 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-service-ca\") pod \"6e4836bb-d56f-437a-893f-83160d58f2d0\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " Jul 09 14:16:07.769571 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.769150 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-console-config\") pod \"6e4836bb-d56f-437a-893f-83160d58f2d0\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " Jul 09 14:16:07.769571 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.769166 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e4836bb-d56f-437a-893f-83160d58f2d0-console-oauth-config\") pod \"6e4836bb-d56f-437a-893f-83160d58f2d0\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " Jul 09 14:16:07.769571 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.769187 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-oauth-serving-cert\") pod \"6e4836bb-d56f-437a-893f-83160d58f2d0\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " Jul 09 14:16:07.769571 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.769340 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e4836bb-d56f-437a-893f-83160d58f2d0-console-serving-cert\") pod \"6e4836bb-d56f-437a-893f-83160d58f2d0\" (UID: \"6e4836bb-d56f-437a-893f-83160d58f2d0\") " Jul 09 14:16:07.769571 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.769510 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6e4836bb-d56f-437a-893f-83160d58f2d0" (UID: "6e4836bb-d56f-437a-893f-83160d58f2d0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:16:07.769804 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.769642 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6e4836bb-d56f-437a-893f-83160d58f2d0" (UID: "6e4836bb-d56f-437a-893f-83160d58f2d0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:16:07.769804 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.769656 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-service-ca" (OuterVolumeSpecName: "service-ca") pod "6e4836bb-d56f-437a-893f-83160d58f2d0" (UID: "6e4836bb-d56f-437a-893f-83160d58f2d0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:16:07.769804 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.769665 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-console-config" (OuterVolumeSpecName: "console-config") pod "6e4836bb-d56f-437a-893f-83160d58f2d0" (UID: "6e4836bb-d56f-437a-893f-83160d58f2d0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:16:07.769804 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.769683 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-trusted-ca-bundle\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:07.771473 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.771446 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e4836bb-d56f-437a-893f-83160d58f2d0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6e4836bb-d56f-437a-893f-83160d58f2d0" (UID: "6e4836bb-d56f-437a-893f-83160d58f2d0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:16:07.771473 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.771466 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e4836bb-d56f-437a-893f-83160d58f2d0-kube-api-access-797f9" (OuterVolumeSpecName: "kube-api-access-797f9") pod "6e4836bb-d56f-437a-893f-83160d58f2d0" (UID: "6e4836bb-d56f-437a-893f-83160d58f2d0"). InnerVolumeSpecName "kube-api-access-797f9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:16:07.771617 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.771485 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e4836bb-d56f-437a-893f-83160d58f2d0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6e4836bb-d56f-437a-893f-83160d58f2d0" (UID: "6e4836bb-d56f-437a-893f-83160d58f2d0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:16:07.871043 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.871009 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6e4836bb-d56f-437a-893f-83160d58f2d0-console-serving-cert\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:07.871173 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.871045 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-797f9\" (UniqueName: \"kubernetes.io/projected/6e4836bb-d56f-437a-893f-83160d58f2d0-kube-api-access-797f9\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:07.871173 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.871060 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-service-ca\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:07.871173 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.871072 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-console-config\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:07.871173 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.871080 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6e4836bb-d56f-437a-893f-83160d58f2d0-console-oauth-config\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:07.871173 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:07.871088 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6e4836bb-d56f-437a-893f-83160d58f2d0-oauth-serving-cert\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:08.453388 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:08.453355 2581 generic.go:356] "Generic (PLEG): container finished" podID="6e4836bb-d56f-437a-893f-83160d58f2d0" containerID="57330ceffd0238800ef37883362cbbe0ef04b47aeffe86627b9ee257006ed556" exitCode=0 Jul 09 14:16:08.453548 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:08.453403 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77d69d764d-fn55z" Jul 09 14:16:08.453548 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:08.453454 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77d69d764d-fn55z" event={"ID":"6e4836bb-d56f-437a-893f-83160d58f2d0","Type":"ContainerDied","Data":"57330ceffd0238800ef37883362cbbe0ef04b47aeffe86627b9ee257006ed556"} Jul 09 14:16:08.453548 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:08.453506 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77d69d764d-fn55z" event={"ID":"6e4836bb-d56f-437a-893f-83160d58f2d0","Type":"ContainerDied","Data":"85af2d3b0648e1f8f3f3e9206c6c1beda68734df46c57f6ed3408ce2b41db3be"} Jul 09 14:16:08.453548 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:08.453531 2581 scope.go:117] "RemoveContainer" containerID="57330ceffd0238800ef37883362cbbe0ef04b47aeffe86627b9ee257006ed556" Jul 09 14:16:08.462260 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:08.462237 2581 scope.go:117] "RemoveContainer" containerID="57330ceffd0238800ef37883362cbbe0ef04b47aeffe86627b9ee257006ed556" Jul 09 14:16:08.462520 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:16:08.462493 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57330ceffd0238800ef37883362cbbe0ef04b47aeffe86627b9ee257006ed556\": container with ID starting with 57330ceffd0238800ef37883362cbbe0ef04b47aeffe86627b9ee257006ed556 not found: ID does not exist" containerID="57330ceffd0238800ef37883362cbbe0ef04b47aeffe86627b9ee257006ed556" Jul 09 14:16:08.462562 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:08.462529 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57330ceffd0238800ef37883362cbbe0ef04b47aeffe86627b9ee257006ed556"} err="failed to get container status \"57330ceffd0238800ef37883362cbbe0ef04b47aeffe86627b9ee257006ed556\": rpc error: code = NotFound desc = could not find container \"57330ceffd0238800ef37883362cbbe0ef04b47aeffe86627b9ee257006ed556\": container with ID starting with 57330ceffd0238800ef37883362cbbe0ef04b47aeffe86627b9ee257006ed556 not found: ID does not exist" Jul 09 14:16:08.475652 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:08.475626 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77d69d764d-fn55z"] Jul 09 14:16:08.481240 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:08.481215 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77d69d764d-fn55z"] Jul 09 14:16:08.976600 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:08.976565 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e4836bb-d56f-437a-893f-83160d58f2d0" path="/var/lib/kubelet/pods/6e4836bb-d56f-437a-893f-83160d58f2d0/volumes" Jul 09 14:16:10.613222 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:10.613187 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:16:10.613625 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:10.613493 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="alertmanager" containerID="cri-o://8c80ba50c05e3e823a7d77b8117288e29560e6315a2238fb88faa7271a62d18d" gracePeriod=120 Jul 09 14:16:10.613625 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:10.613550 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="kube-rbac-proxy-metric" containerID="cri-o://478280f388173af8f995287de6eccb35c078ae2c0b09df268cf71c5353b5e3bf" gracePeriod=120 Jul 09 14:16:10.613625 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:10.613589 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="prom-label-proxy" containerID="cri-o://a66f59b512bb401420d7d3398d8af76c960b052ef282b96b42b3fc77f561756d" gracePeriod=120 Jul 09 14:16:10.613789 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:10.613627 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="kube-rbac-proxy-web" containerID="cri-o://912dab06057c1a1b893ac0b035f98427835c9139e850a926195866e2e6e5c859" gracePeriod=120 Jul 09 14:16:10.613789 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:10.613644 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="config-reloader" containerID="cri-o://ae36e920e1c27d0c2c1b143e9e873a46e2e25550f37425eda105c384d5380176" gracePeriod=120 Jul 09 14:16:10.613789 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:10.613657 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="kube-rbac-proxy" containerID="cri-o://fbdfcb9d53f1e511fff41b18b66c0820bb6d41154fedc3f7b4a5cf3d90a75879" gracePeriod=120 Jul 09 14:16:11.470307 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:11.470275 2581 generic.go:356] "Generic (PLEG): container finished" podID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerID="a66f59b512bb401420d7d3398d8af76c960b052ef282b96b42b3fc77f561756d" exitCode=0 Jul 09 14:16:11.470307 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:11.470301 2581 generic.go:356] "Generic (PLEG): container finished" podID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerID="fbdfcb9d53f1e511fff41b18b66c0820bb6d41154fedc3f7b4a5cf3d90a75879" exitCode=0 Jul 09 14:16:11.470307 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:11.470307 2581 generic.go:356] "Generic (PLEG): container finished" podID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerID="912dab06057c1a1b893ac0b035f98427835c9139e850a926195866e2e6e5c859" exitCode=0 Jul 09 14:16:11.470307 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:11.470313 2581 generic.go:356] "Generic (PLEG): container finished" podID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerID="ae36e920e1c27d0c2c1b143e9e873a46e2e25550f37425eda105c384d5380176" exitCode=0 Jul 09 14:16:11.470307 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:11.470320 2581 generic.go:356] "Generic (PLEG): container finished" podID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerID="8c80ba50c05e3e823a7d77b8117288e29560e6315a2238fb88faa7271a62d18d" exitCode=0 Jul 09 14:16:11.470683 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:11.470347 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerDied","Data":"a66f59b512bb401420d7d3398d8af76c960b052ef282b96b42b3fc77f561756d"} Jul 09 14:16:11.470683 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:11.470409 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerDied","Data":"fbdfcb9d53f1e511fff41b18b66c0820bb6d41154fedc3f7b4a5cf3d90a75879"} Jul 09 14:16:11.470683 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:11.470438 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerDied","Data":"912dab06057c1a1b893ac0b035f98427835c9139e850a926195866e2e6e5c859"} Jul 09 14:16:11.470683 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:11.470462 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerDied","Data":"ae36e920e1c27d0c2c1b143e9e873a46e2e25550f37425eda105c384d5380176"} Jul 09 14:16:11.470683 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:11.470487 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerDied","Data":"8c80ba50c05e3e823a7d77b8117288e29560e6315a2238fb88faa7271a62d18d"} Jul 09 14:16:11.858917 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:11.858883 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.007803 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.007702 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy\") pod \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " Jul 09 14:16:12.007803 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.007750 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-config-volume\") pod \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " Jul 09 14:16:12.007803 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.007783 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy-web\") pod \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " Jul 09 14:16:12.008114 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.007815 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/444dcf65-3841-445f-ac9e-6f69b1c35fa1-tls-assets\") pod \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " Jul 09 14:16:12.008114 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.007845 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-main-tls\") pod \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " Jul 09 14:16:12.008114 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.007932 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " Jul 09 14:16:12.008114 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.008008 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-main-db\") pod \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " Jul 09 14:16:12.008114 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.008045 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gj4xw\" (UniqueName: \"kubernetes.io/projected/444dcf65-3841-445f-ac9e-6f69b1c35fa1-kube-api-access-gj4xw\") pod \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " Jul 09 14:16:12.008114 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.008078 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-web-config\") pod \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " Jul 09 14:16:12.008422 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.008121 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-cluster-tls-config\") pod \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " Jul 09 14:16:12.008422 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.008154 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/444dcf65-3841-445f-ac9e-6f69b1c35fa1-config-out\") pod \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " Jul 09 14:16:12.008422 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.008180 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-trusted-ca-bundle\") pod \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " Jul 09 14:16:12.008422 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.008246 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-metrics-client-ca\") pod \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\" (UID: \"444dcf65-3841-445f-ac9e-6f69b1c35fa1\") " Jul 09 14:16:12.009074 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.009013 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "444dcf65-3841-445f-ac9e-6f69b1c35fa1" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:16:12.009199 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.009107 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "444dcf65-3841-445f-ac9e-6f69b1c35fa1" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:16:12.009513 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.009474 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "444dcf65-3841-445f-ac9e-6f69b1c35fa1" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:16:12.011216 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.011188 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "444dcf65-3841-445f-ac9e-6f69b1c35fa1" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:16:12.011587 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.011547 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "444dcf65-3841-445f-ac9e-6f69b1c35fa1" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:16:12.011782 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.011761 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-config-volume" (OuterVolumeSpecName: "config-volume") pod "444dcf65-3841-445f-ac9e-6f69b1c35fa1" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:16:12.011930 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.011907 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/444dcf65-3841-445f-ac9e-6f69b1c35fa1-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "444dcf65-3841-445f-ac9e-6f69b1c35fa1" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:16:12.012272 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.012250 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/444dcf65-3841-445f-ac9e-6f69b1c35fa1-kube-api-access-gj4xw" (OuterVolumeSpecName: "kube-api-access-gj4xw") pod "444dcf65-3841-445f-ac9e-6f69b1c35fa1" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1"). InnerVolumeSpecName "kube-api-access-gj4xw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:16:12.012343 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.012251 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "444dcf65-3841-445f-ac9e-6f69b1c35fa1" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:16:12.012411 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.012390 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "444dcf65-3841-445f-ac9e-6f69b1c35fa1" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:16:12.012715 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.012696 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/444dcf65-3841-445f-ac9e-6f69b1c35fa1-config-out" (OuterVolumeSpecName: "config-out") pod "444dcf65-3841-445f-ac9e-6f69b1c35fa1" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:16:12.015727 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.015705 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "444dcf65-3841-445f-ac9e-6f69b1c35fa1" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:16:12.022481 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.022439 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-web-config" (OuterVolumeSpecName: "web-config") pod "444dcf65-3841-445f-ac9e-6f69b1c35fa1" (UID: "444dcf65-3841-445f-ac9e-6f69b1c35fa1"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:16:12.109345 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.109312 2581 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-metrics-client-ca\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:12.109345 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.109341 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:12.109345 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.109352 2581 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:12.109580 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.109362 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:12.109580 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.109439 2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/444dcf65-3841-445f-ac9e-6f69b1c35fa1-tls-assets\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:12.109580 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.109450 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-main-tls\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:12.109580 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.109458 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:12.109580 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.109470 2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-main-db\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:12.109580 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.109478 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj4xw\" (UniqueName: \"kubernetes.io/projected/444dcf65-3841-445f-ac9e-6f69b1c35fa1-kube-api-access-gj4xw\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:12.109580 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.109486 2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-web-config\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:12.109580 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.109496 2581 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/444dcf65-3841-445f-ac9e-6f69b1c35fa1-cluster-tls-config\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:12.109580 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.109503 2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/444dcf65-3841-445f-ac9e-6f69b1c35fa1-config-out\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:12.109580 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.109513 2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/444dcf65-3841-445f-ac9e-6f69b1c35fa1-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:12.477268 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.477232 2581 generic.go:356] "Generic (PLEG): container finished" podID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerID="478280f388173af8f995287de6eccb35c078ae2c0b09df268cf71c5353b5e3bf" exitCode=0 Jul 09 14:16:12.477452 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.477327 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.477452 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.477320 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerDied","Data":"478280f388173af8f995287de6eccb35c078ae2c0b09df268cf71c5353b5e3bf"} Jul 09 14:16:12.477574 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.477525 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"444dcf65-3841-445f-ac9e-6f69b1c35fa1","Type":"ContainerDied","Data":"0fd1d03df825a545f2960d6db41f6e6aee10c6d253840f9dfc53e4633ba6efeb"} Jul 09 14:16:12.477618 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.477584 2581 scope.go:117] "RemoveContainer" containerID="a66f59b512bb401420d7d3398d8af76c960b052ef282b96b42b3fc77f561756d" Jul 09 14:16:12.485368 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.485251 2581 scope.go:117] "RemoveContainer" containerID="478280f388173af8f995287de6eccb35c078ae2c0b09df268cf71c5353b5e3bf" Jul 09 14:16:12.492348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.492332 2581 scope.go:117] "RemoveContainer" containerID="fbdfcb9d53f1e511fff41b18b66c0820bb6d41154fedc3f7b4a5cf3d90a75879" Jul 09 14:16:12.500745 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.500728 2581 scope.go:117] "RemoveContainer" containerID="912dab06057c1a1b893ac0b035f98427835c9139e850a926195866e2e6e5c859" Jul 09 14:16:12.502934 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.502912 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:16:12.507116 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.507095 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:16:12.508190 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.508176 2581 scope.go:117] "RemoveContainer" containerID="ae36e920e1c27d0c2c1b143e9e873a46e2e25550f37425eda105c384d5380176" Jul 09 14:16:12.514672 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.514655 2581 scope.go:117] "RemoveContainer" containerID="8c80ba50c05e3e823a7d77b8117288e29560e6315a2238fb88faa7271a62d18d" Jul 09 14:16:12.521221 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.521207 2581 scope.go:117] "RemoveContainer" containerID="5bdaa968cf3264e038afc6acfb41364b0271547edf717c1a31e2efd727babf7d" Jul 09 14:16:12.527909 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.527892 2581 scope.go:117] "RemoveContainer" containerID="a66f59b512bb401420d7d3398d8af76c960b052ef282b96b42b3fc77f561756d" Jul 09 14:16:12.528199 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:16:12.528182 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a66f59b512bb401420d7d3398d8af76c960b052ef282b96b42b3fc77f561756d\": container with ID starting with a66f59b512bb401420d7d3398d8af76c960b052ef282b96b42b3fc77f561756d not found: ID does not exist" containerID="a66f59b512bb401420d7d3398d8af76c960b052ef282b96b42b3fc77f561756d" Jul 09 14:16:12.528264 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.528206 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a66f59b512bb401420d7d3398d8af76c960b052ef282b96b42b3fc77f561756d"} err="failed to get container status \"a66f59b512bb401420d7d3398d8af76c960b052ef282b96b42b3fc77f561756d\": rpc error: code = NotFound desc = could not find container \"a66f59b512bb401420d7d3398d8af76c960b052ef282b96b42b3fc77f561756d\": container with ID starting with a66f59b512bb401420d7d3398d8af76c960b052ef282b96b42b3fc77f561756d not found: ID does not exist" Jul 09 14:16:12.528264 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.528224 2581 scope.go:117] "RemoveContainer" containerID="478280f388173af8f995287de6eccb35c078ae2c0b09df268cf71c5353b5e3bf" Jul 09 14:16:12.528498 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:16:12.528479 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"478280f388173af8f995287de6eccb35c078ae2c0b09df268cf71c5353b5e3bf\": container with ID starting with 478280f388173af8f995287de6eccb35c078ae2c0b09df268cf71c5353b5e3bf not found: ID does not exist" containerID="478280f388173af8f995287de6eccb35c078ae2c0b09df268cf71c5353b5e3bf" Jul 09 14:16:12.528559 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.528504 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"478280f388173af8f995287de6eccb35c078ae2c0b09df268cf71c5353b5e3bf"} err="failed to get container status \"478280f388173af8f995287de6eccb35c078ae2c0b09df268cf71c5353b5e3bf\": rpc error: code = NotFound desc = could not find container \"478280f388173af8f995287de6eccb35c078ae2c0b09df268cf71c5353b5e3bf\": container with ID starting with 478280f388173af8f995287de6eccb35c078ae2c0b09df268cf71c5353b5e3bf not found: ID does not exist" Jul 09 14:16:12.528559 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.528523 2581 scope.go:117] "RemoveContainer" containerID="fbdfcb9d53f1e511fff41b18b66c0820bb6d41154fedc3f7b4a5cf3d90a75879" Jul 09 14:16:12.528772 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:16:12.528747 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbdfcb9d53f1e511fff41b18b66c0820bb6d41154fedc3f7b4a5cf3d90a75879\": container with ID starting with fbdfcb9d53f1e511fff41b18b66c0820bb6d41154fedc3f7b4a5cf3d90a75879 not found: ID does not exist" containerID="fbdfcb9d53f1e511fff41b18b66c0820bb6d41154fedc3f7b4a5cf3d90a75879" Jul 09 14:16:12.528814 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.528769 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbdfcb9d53f1e511fff41b18b66c0820bb6d41154fedc3f7b4a5cf3d90a75879"} err="failed to get container status \"fbdfcb9d53f1e511fff41b18b66c0820bb6d41154fedc3f7b4a5cf3d90a75879\": rpc error: code = NotFound desc = could not find container \"fbdfcb9d53f1e511fff41b18b66c0820bb6d41154fedc3f7b4a5cf3d90a75879\": container with ID starting with fbdfcb9d53f1e511fff41b18b66c0820bb6d41154fedc3f7b4a5cf3d90a75879 not found: ID does not exist" Jul 09 14:16:12.528814 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.528785 2581 scope.go:117] "RemoveContainer" containerID="912dab06057c1a1b893ac0b035f98427835c9139e850a926195866e2e6e5c859" Jul 09 14:16:12.529014 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:16:12.528998 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"912dab06057c1a1b893ac0b035f98427835c9139e850a926195866e2e6e5c859\": container with ID starting with 912dab06057c1a1b893ac0b035f98427835c9139e850a926195866e2e6e5c859 not found: ID does not exist" containerID="912dab06057c1a1b893ac0b035f98427835c9139e850a926195866e2e6e5c859" Jul 09 14:16:12.529048 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.529020 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"912dab06057c1a1b893ac0b035f98427835c9139e850a926195866e2e6e5c859"} err="failed to get container status \"912dab06057c1a1b893ac0b035f98427835c9139e850a926195866e2e6e5c859\": rpc error: code = NotFound desc = could not find container \"912dab06057c1a1b893ac0b035f98427835c9139e850a926195866e2e6e5c859\": container with ID starting with 912dab06057c1a1b893ac0b035f98427835c9139e850a926195866e2e6e5c859 not found: ID does not exist" Jul 09 14:16:12.529048 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.529035 2581 scope.go:117] "RemoveContainer" containerID="ae36e920e1c27d0c2c1b143e9e873a46e2e25550f37425eda105c384d5380176" Jul 09 14:16:12.529266 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:16:12.529250 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae36e920e1c27d0c2c1b143e9e873a46e2e25550f37425eda105c384d5380176\": container with ID starting with ae36e920e1c27d0c2c1b143e9e873a46e2e25550f37425eda105c384d5380176 not found: ID does not exist" containerID="ae36e920e1c27d0c2c1b143e9e873a46e2e25550f37425eda105c384d5380176" Jul 09 14:16:12.529316 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.529272 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae36e920e1c27d0c2c1b143e9e873a46e2e25550f37425eda105c384d5380176"} err="failed to get container status \"ae36e920e1c27d0c2c1b143e9e873a46e2e25550f37425eda105c384d5380176\": rpc error: code = NotFound desc = could not find container \"ae36e920e1c27d0c2c1b143e9e873a46e2e25550f37425eda105c384d5380176\": container with ID starting with ae36e920e1c27d0c2c1b143e9e873a46e2e25550f37425eda105c384d5380176 not found: ID does not exist" Jul 09 14:16:12.529316 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.529285 2581 scope.go:117] "RemoveContainer" containerID="8c80ba50c05e3e823a7d77b8117288e29560e6315a2238fb88faa7271a62d18d" Jul 09 14:16:12.529486 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:16:12.529467 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c80ba50c05e3e823a7d77b8117288e29560e6315a2238fb88faa7271a62d18d\": container with ID starting with 8c80ba50c05e3e823a7d77b8117288e29560e6315a2238fb88faa7271a62d18d not found: ID does not exist" containerID="8c80ba50c05e3e823a7d77b8117288e29560e6315a2238fb88faa7271a62d18d" Jul 09 14:16:12.529563 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.529495 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c80ba50c05e3e823a7d77b8117288e29560e6315a2238fb88faa7271a62d18d"} err="failed to get container status \"8c80ba50c05e3e823a7d77b8117288e29560e6315a2238fb88faa7271a62d18d\": rpc error: code = NotFound desc = could not find container \"8c80ba50c05e3e823a7d77b8117288e29560e6315a2238fb88faa7271a62d18d\": container with ID starting with 8c80ba50c05e3e823a7d77b8117288e29560e6315a2238fb88faa7271a62d18d not found: ID does not exist" Jul 09 14:16:12.529563 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.529516 2581 scope.go:117] "RemoveContainer" containerID="5bdaa968cf3264e038afc6acfb41364b0271547edf717c1a31e2efd727babf7d" Jul 09 14:16:12.529726 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:16:12.529710 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bdaa968cf3264e038afc6acfb41364b0271547edf717c1a31e2efd727babf7d\": container with ID starting with 5bdaa968cf3264e038afc6acfb41364b0271547edf717c1a31e2efd727babf7d not found: ID does not exist" containerID="5bdaa968cf3264e038afc6acfb41364b0271547edf717c1a31e2efd727babf7d" Jul 09 14:16:12.529767 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.529730 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bdaa968cf3264e038afc6acfb41364b0271547edf717c1a31e2efd727babf7d"} err="failed to get container status \"5bdaa968cf3264e038afc6acfb41364b0271547edf717c1a31e2efd727babf7d\": rpc error: code = NotFound desc = could not find container \"5bdaa968cf3264e038afc6acfb41364b0271547edf717c1a31e2efd727babf7d\": container with ID starting with 5bdaa968cf3264e038afc6acfb41364b0271547edf717c1a31e2efd727babf7d not found: ID does not exist" Jul 09 14:16:12.533757 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.533736 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:16:12.534108 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534091 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e4836bb-d56f-437a-893f-83160d58f2d0" containerName="console" Jul 09 14:16:12.534182 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534110 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e4836bb-d56f-437a-893f-83160d58f2d0" containerName="console" Jul 09 14:16:12.534182 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534152 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="kube-rbac-proxy-web" Jul 09 14:16:12.534182 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534161 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="kube-rbac-proxy-web" Jul 09 14:16:12.534182 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534170 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="prom-label-proxy" Jul 09 14:16:12.534182 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534177 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="prom-label-proxy" Jul 09 14:16:12.534325 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534187 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="init-config-reloader" Jul 09 14:16:12.534325 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534195 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="init-config-reloader" Jul 09 14:16:12.534325 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534206 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="kube-rbac-proxy" Jul 09 14:16:12.534325 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534213 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="kube-rbac-proxy" Jul 09 14:16:12.534325 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534229 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="alertmanager" Jul 09 14:16:12.534325 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534237 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="alertmanager" Jul 09 14:16:12.534325 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534246 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="config-reloader" Jul 09 14:16:12.534325 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534255 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="config-reloader" Jul 09 14:16:12.534325 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534265 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="kube-rbac-proxy-metric" Jul 09 14:16:12.534325 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534270 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="kube-rbac-proxy-metric" Jul 09 14:16:12.534325 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534324 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="kube-rbac-proxy-metric" Jul 09 14:16:12.534619 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534332 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="alertmanager" Jul 09 14:16:12.534619 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534339 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="config-reloader" Jul 09 14:16:12.534619 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534344 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="kube-rbac-proxy" Jul 09 14:16:12.534619 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534350 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e4836bb-d56f-437a-893f-83160d58f2d0" containerName="console" Jul 09 14:16:12.534619 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534357 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="prom-label-proxy" Jul 09 14:16:12.534619 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.534362 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" containerName="kube-rbac-proxy-web" Jul 09 14:16:12.539224 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.539208 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.542472 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.542455 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 14:16:12.542603 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.542456 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 14:16:12.542796 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.542749 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 14:16:12.542796 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.542784 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 14:16:12.542974 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.542830 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 14:16:12.542974 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.542830 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 14:16:12.542974 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.542921 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dx5nj\"" Jul 09 14:16:12.543165 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.542976 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 14:16:12.543165 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.543064 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 14:16:12.547943 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.547923 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 14:16:12.554438 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.554414 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:16:12.614315 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.614275 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.614453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.614323 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.614453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.614346 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.614453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.614362 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.614453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.614383 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-web-config\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.614453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.614407 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.614623 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.614465 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-config-out\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.614623 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.614491 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.614623 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.614519 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.614623 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.614547 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-config-volume\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.614623 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.614577 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t69gj\" (UniqueName: \"kubernetes.io/projected/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-kube-api-access-t69gj\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.614772 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.614643 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.614772 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.614662 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.715065 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.715029 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.715212 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.715077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.715212 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.715109 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.715212 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.715132 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.715212 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.715159 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-web-config\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.715212 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.715192 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.715520 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.715220 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-config-out\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.715520 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.715243 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.715520 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.715395 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.715520 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.715437 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-config-volume\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.715520 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.715459 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t69gj\" (UniqueName: \"kubernetes.io/projected/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-kube-api-access-t69gj\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.715520 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.715492 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.715520 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.715519 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.716857 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.716186 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.717338 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.715495 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.718015 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.717990 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.718323 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.718301 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-config-out\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.718750 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.718723 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.718854 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.718725 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.719000 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.718976 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.719480 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.719455 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.719669 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.719650 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-config-volume\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.719710 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.719663 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-web-config\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.719830 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.719811 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.720153 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.720137 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.725226 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.725208 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t69gj\" (UniqueName: \"kubernetes.io/projected/571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a-kube-api-access-t69gj\") pod \"alertmanager-main-0\" (UID: \"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.848515 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.848487 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:16:12.977903 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.977868 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="444dcf65-3841-445f-ac9e-6f69b1c35fa1" path="/var/lib/kubelet/pods/444dcf65-3841-445f-ac9e-6f69b1c35fa1/volumes" Jul 09 14:16:12.996858 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:12.996829 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:16:13.000117 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:16:13.000088 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod571f0ace_8ae8_4040_8b3f_5a48e6c5bb3a.slice/crio-6b15377b11c3e656888e31068fa0341a3914992d16b6d04e0996dcb31060e599 WatchSource:0}: Error finding container 6b15377b11c3e656888e31068fa0341a3914992d16b6d04e0996dcb31060e599: Status 404 returned error can't find the container with id 6b15377b11c3e656888e31068fa0341a3914992d16b6d04e0996dcb31060e599 Jul 09 14:16:13.483521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:13.483422 2581 generic.go:356] "Generic (PLEG): container finished" podID="571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a" containerID="6cf510092509beb87878a45c71f7b213347b6dfd9cdd4c7f19aed76c210af19a" exitCode=0 Jul 09 14:16:13.483674 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:13.483512 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a","Type":"ContainerDied","Data":"6cf510092509beb87878a45c71f7b213347b6dfd9cdd4c7f19aed76c210af19a"} Jul 09 14:16:13.483674 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:13.483587 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a","Type":"ContainerStarted","Data":"6b15377b11c3e656888e31068fa0341a3914992d16b6d04e0996dcb31060e599"} Jul 09 14:16:14.491059 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:14.491020 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a","Type":"ContainerStarted","Data":"49dbf9535dc4d6acef27ba392067dfd7f16d85b1757ec213ef53facdc7f8e274"} Jul 09 14:16:14.491424 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:14.491116 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a","Type":"ContainerStarted","Data":"9ec1530847e121a9b7e768fc8d7de1d8b1c2f9f8c17fcf1f3cc7d0da0b86e8bb"} Jul 09 14:16:14.491424 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:14.491149 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a","Type":"ContainerStarted","Data":"a0c8a6b1156f0a97ed44fc11d186609a88aa60ea08481b1996c66987194b8ad2"} Jul 09 14:16:14.491424 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:14.491176 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a","Type":"ContainerStarted","Data":"cfe936db2b81dc6d2be420fd15032e5e05e385f2bfdaaea3ed2c881573a336d8"} Jul 09 14:16:14.491424 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:14.491206 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a","Type":"ContainerStarted","Data":"fa6c30c0d954bd0e71208d530271953d080c1eb38afcd3244c85ae427f687692"} Jul 09 14:16:14.491424 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:14.491231 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a","Type":"ContainerStarted","Data":"3fb24fca14d474ccaa7a9f8ac495c7b22e3bb1673da16bb996ddc002167a1a14"} Jul 09 14:16:16.946824 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:16.946771 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:16.947260 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:16.946860 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:16.951756 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:16.951732 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:16.982316 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:16.982269 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.966057007 podStartE2EDuration="4.982250798s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.193791ms" isStatefulPod=false podCreationTimestamp="2026-07-09 14:16:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:16:14.542329125 +0000 UTC m=+94.665468508" watchObservedRunningTime="2026-07-09 14:16:16.982250798 +0000 UTC m=+97.105390182" Jul 09 14:16:17.507357 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:17.507329 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:16:17.560737 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:17.560708 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb99b5996-b4nh6"] Jul 09 14:16:23.390550 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:23.390518 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qqzlh" Jul 09 14:16:42.580854 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.580788 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5cb99b5996-b4nh6" podUID="8589e22a-3e4a-4b1a-9427-7ed75f6514dc" containerName="console" containerID="cri-o://109cb26ad2306de16d05a9314a4a5bf64acbd572cc817e282d6652acf099cb6f" gracePeriod=15 Jul 09 14:16:42.834910 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.834851 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:16:42.969757 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.969720 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-serving-cert\") pod \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " Jul 09 14:16:42.969922 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.969799 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-service-ca\") pod \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " Jul 09 14:16:42.969922 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.969826 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-trusted-ca-bundle\") pod \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " Jul 09 14:16:42.969922 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.969848 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-oauth-serving-cert\") pod \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " Jul 09 14:16:42.970080 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.969920 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wzkg\" (UniqueName: \"kubernetes.io/projected/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-kube-api-access-5wzkg\") pod \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " Jul 09 14:16:42.970080 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.969983 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-oauth-config\") pod \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " Jul 09 14:16:42.970080 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.970011 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-config\") pod \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\" (UID: \"8589e22a-3e4a-4b1a-9427-7ed75f6514dc\") " Jul 09 14:16:42.970296 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.970264 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8589e22a-3e4a-4b1a-9427-7ed75f6514dc" (UID: "8589e22a-3e4a-4b1a-9427-7ed75f6514dc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:16:42.970355 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.970286 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8589e22a-3e4a-4b1a-9427-7ed75f6514dc" (UID: "8589e22a-3e4a-4b1a-9427-7ed75f6514dc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:16:42.970513 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.970491 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-config" (OuterVolumeSpecName: "console-config") pod "8589e22a-3e4a-4b1a-9427-7ed75f6514dc" (UID: "8589e22a-3e4a-4b1a-9427-7ed75f6514dc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:16:42.970584 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.970550 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-service-ca" (OuterVolumeSpecName: "service-ca") pod "8589e22a-3e4a-4b1a-9427-7ed75f6514dc" (UID: "8589e22a-3e4a-4b1a-9427-7ed75f6514dc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:16:42.972308 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.972282 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8589e22a-3e4a-4b1a-9427-7ed75f6514dc" (UID: "8589e22a-3e4a-4b1a-9427-7ed75f6514dc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:16:42.972503 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.972482 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-kube-api-access-5wzkg" (OuterVolumeSpecName: "kube-api-access-5wzkg") pod "8589e22a-3e4a-4b1a-9427-7ed75f6514dc" (UID: "8589e22a-3e4a-4b1a-9427-7ed75f6514dc"). InnerVolumeSpecName "kube-api-access-5wzkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:16:42.972549 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:42.972525 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8589e22a-3e4a-4b1a-9427-7ed75f6514dc" (UID: "8589e22a-3e4a-4b1a-9427-7ed75f6514dc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:16:43.070917 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.070876 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-serving-cert\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:43.070917 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.070911 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-service-ca\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:43.070917 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.070921 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-trusted-ca-bundle\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:43.071155 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.070930 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-oauth-serving-cert\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:43.071155 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.070940 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wzkg\" (UniqueName: \"kubernetes.io/projected/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-kube-api-access-5wzkg\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:43.071155 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.070966 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-oauth-config\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:43.071155 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.070977 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8589e22a-3e4a-4b1a-9427-7ed75f6514dc-console-config\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:16:43.603055 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.603021 2581 generic.go:356] "Generic (PLEG): container finished" podID="8589e22a-3e4a-4b1a-9427-7ed75f6514dc" containerID="109cb26ad2306de16d05a9314a4a5bf64acbd572cc817e282d6652acf099cb6f" exitCode=0 Jul 09 14:16:43.603510 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.603068 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb99b5996-b4nh6" Jul 09 14:16:43.603510 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.603106 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb99b5996-b4nh6" event={"ID":"8589e22a-3e4a-4b1a-9427-7ed75f6514dc","Type":"ContainerDied","Data":"109cb26ad2306de16d05a9314a4a5bf64acbd572cc817e282d6652acf099cb6f"} Jul 09 14:16:43.603510 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.603166 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb99b5996-b4nh6" event={"ID":"8589e22a-3e4a-4b1a-9427-7ed75f6514dc","Type":"ContainerDied","Data":"9de3a23db0e66499b434e187c66b9d0ff6e646d0a98fae538397eedfdf870172"} Jul 09 14:16:43.603510 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.603194 2581 scope.go:117] "RemoveContainer" containerID="109cb26ad2306de16d05a9314a4a5bf64acbd572cc817e282d6652acf099cb6f" Jul 09 14:16:43.611112 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.611093 2581 scope.go:117] "RemoveContainer" containerID="109cb26ad2306de16d05a9314a4a5bf64acbd572cc817e282d6652acf099cb6f" Jul 09 14:16:43.611359 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:16:43.611339 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"109cb26ad2306de16d05a9314a4a5bf64acbd572cc817e282d6652acf099cb6f\": container with ID starting with 109cb26ad2306de16d05a9314a4a5bf64acbd572cc817e282d6652acf099cb6f not found: ID does not exist" containerID="109cb26ad2306de16d05a9314a4a5bf64acbd572cc817e282d6652acf099cb6f" Jul 09 14:16:43.611431 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.611371 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"109cb26ad2306de16d05a9314a4a5bf64acbd572cc817e282d6652acf099cb6f"} err="failed to get container status \"109cb26ad2306de16d05a9314a4a5bf64acbd572cc817e282d6652acf099cb6f\": rpc error: code = NotFound desc = could not find container \"109cb26ad2306de16d05a9314a4a5bf64acbd572cc817e282d6652acf099cb6f\": container with ID starting with 109cb26ad2306de16d05a9314a4a5bf64acbd572cc817e282d6652acf099cb6f not found: ID does not exist" Jul 09 14:16:43.622154 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.622125 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb99b5996-b4nh6"] Jul 09 14:16:43.624222 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:43.624199 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cb99b5996-b4nh6"] Jul 09 14:16:44.981536 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:16:44.981502 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8589e22a-3e4a-4b1a-9427-7ed75f6514dc" path="/var/lib/kubelet/pods/8589e22a-3e4a-4b1a-9427-7ed75f6514dc/volumes" Jul 09 14:17:01.693681 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.693647 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6"] Jul 09 14:17:01.694184 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.694095 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8589e22a-3e4a-4b1a-9427-7ed75f6514dc" containerName="console" Jul 09 14:17:01.694184 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.694114 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8589e22a-3e4a-4b1a-9427-7ed75f6514dc" containerName="console" Jul 09 14:17:01.694298 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.694194 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8589e22a-3e4a-4b1a-9427-7ed75f6514dc" containerName="console" Jul 09 14:17:01.697284 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.697262 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" Jul 09 14:17:01.700627 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.700578 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 14:17:01.700771 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.700753 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 14:17:01.701995 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.701981 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tr2j5\"" Jul 09 14:17:01.706851 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.706829 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6"] Jul 09 14:17:01.727391 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.727359 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6\" (UID: \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" Jul 09 14:17:01.727546 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.727406 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-766xk\" (UniqueName: \"kubernetes.io/projected/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-kube-api-access-766xk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6\" (UID: \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" Jul 09 14:17:01.727546 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.727442 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6\" (UID: \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" Jul 09 14:17:01.828460 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.828420 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6\" (UID: \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" Jul 09 14:17:01.828577 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.828472 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-766xk\" (UniqueName: \"kubernetes.io/projected/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-kube-api-access-766xk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6\" (UID: \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" Jul 09 14:17:01.828577 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.828496 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6\" (UID: \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" Jul 09 14:17:01.828807 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.828787 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6\" (UID: \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" Jul 09 14:17:01.828844 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.828815 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6\" (UID: \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" Jul 09 14:17:01.837746 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:01.837722 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-766xk\" (UniqueName: \"kubernetes.io/projected/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-kube-api-access-766xk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6\" (UID: \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" Jul 09 14:17:02.006806 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:02.006709 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" Jul 09 14:17:02.137916 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:02.137886 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6"] Jul 09 14:17:02.140746 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:17:02.140717 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd8f160a_4f21_4cab_aece_49dd0fe9a3bb.slice/crio-28bafb908e17bf5294b9576b245769b1f0344a5cf162398343435436209e053f WatchSource:0}: Error finding container 28bafb908e17bf5294b9576b245769b1f0344a5cf162398343435436209e053f: Status 404 returned error can't find the container with id 28bafb908e17bf5294b9576b245769b1f0344a5cf162398343435436209e053f Jul 09 14:17:02.678043 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:02.678002 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" event={"ID":"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb","Type":"ContainerStarted","Data":"28bafb908e17bf5294b9576b245769b1f0344a5cf162398343435436209e053f"} Jul 09 14:17:09.710334 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:09.710292 2581 generic.go:356] "Generic (PLEG): container finished" podID="fd8f160a-4f21-4cab-aece-49dd0fe9a3bb" containerID="48da4abdacba1e30786bb1a08f95d98120634cde13a89cd68107094df9d016bd" exitCode=0 Jul 09 14:17:09.710746 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:09.710385 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" event={"ID":"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb","Type":"ContainerDied","Data":"48da4abdacba1e30786bb1a08f95d98120634cde13a89cd68107094df9d016bd"} Jul 09 14:17:11.719399 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:11.719366 2581 generic.go:356] "Generic (PLEG): container finished" podID="fd8f160a-4f21-4cab-aece-49dd0fe9a3bb" containerID="9e2abc7460c818ab4c5777cad2ec2afcceb16e8a2b999371cdfdd588457bac3c" exitCode=0 Jul 09 14:17:11.719810 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:11.719457 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" event={"ID":"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb","Type":"ContainerDied","Data":"9e2abc7460c818ab4c5777cad2ec2afcceb16e8a2b999371cdfdd588457bac3c"} Jul 09 14:17:21.765091 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:21.765056 2581 generic.go:356] "Generic (PLEG): container finished" podID="fd8f160a-4f21-4cab-aece-49dd0fe9a3bb" containerID="1b542993c3957598cee54b48346c6c226ec45dd2b19fcb63a7bb9e4574436862" exitCode=0 Jul 09 14:17:21.765525 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:21.765145 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" event={"ID":"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb","Type":"ContainerDied","Data":"1b542993c3957598cee54b48346c6c226ec45dd2b19fcb63a7bb9e4574436862"} Jul 09 14:17:22.894016 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:22.893990 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" Jul 09 14:17:23.018705 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:23.018672 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-bundle\") pod \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\" (UID: \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\") " Jul 09 14:17:23.018868 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:23.018723 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-766xk\" (UniqueName: \"kubernetes.io/projected/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-kube-api-access-766xk\") pod \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\" (UID: \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\") " Jul 09 14:17:23.018868 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:23.018778 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-util\") pod \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\" (UID: \"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb\") " Jul 09 14:17:23.019650 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:23.019589 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-bundle" (OuterVolumeSpecName: "bundle") pod "fd8f160a-4f21-4cab-aece-49dd0fe9a3bb" (UID: "fd8f160a-4f21-4cab-aece-49dd0fe9a3bb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:17:23.020996 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:23.020967 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-kube-api-access-766xk" (OuterVolumeSpecName: "kube-api-access-766xk") pod "fd8f160a-4f21-4cab-aece-49dd0fe9a3bb" (UID: "fd8f160a-4f21-4cab-aece-49dd0fe9a3bb"). InnerVolumeSpecName "kube-api-access-766xk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:17:23.024669 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:23.024570 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-util" (OuterVolumeSpecName: "util") pod "fd8f160a-4f21-4cab-aece-49dd0fe9a3bb" (UID: "fd8f160a-4f21-4cab-aece-49dd0fe9a3bb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:17:23.119434 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:23.119395 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-bundle\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:17:23.119434 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:23.119427 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-766xk\" (UniqueName: \"kubernetes.io/projected/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-kube-api-access-766xk\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:17:23.119434 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:23.119437 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd8f160a-4f21-4cab-aece-49dd0fe9a3bb-util\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:17:23.774509 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:23.774470 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" event={"ID":"fd8f160a-4f21-4cab-aece-49dd0fe9a3bb","Type":"ContainerDied","Data":"28bafb908e17bf5294b9576b245769b1f0344a5cf162398343435436209e053f"} Jul 09 14:17:23.774509 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:23.774513 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405ktw6" Jul 09 14:17:23.774714 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:23.774546 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28bafb908e17bf5294b9576b245769b1f0344a5cf162398343435436209e053f" Jul 09 14:17:29.899530 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.899493 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n7h8m"] Jul 09 14:17:29.900137 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.899925 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd8f160a-4f21-4cab-aece-49dd0fe9a3bb" containerName="pull" Jul 09 14:17:29.900137 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.899943 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd8f160a-4f21-4cab-aece-49dd0fe9a3bb" containerName="pull" Jul 09 14:17:29.900137 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.899978 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd8f160a-4f21-4cab-aece-49dd0fe9a3bb" containerName="util" Jul 09 14:17:29.900137 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.899988 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd8f160a-4f21-4cab-aece-49dd0fe9a3bb" containerName="util" Jul 09 14:17:29.900137 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.899998 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd8f160a-4f21-4cab-aece-49dd0fe9a3bb" containerName="extract" Jul 09 14:17:29.900137 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.900007 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd8f160a-4f21-4cab-aece-49dd0fe9a3bb" containerName="extract" Jul 09 14:17:29.900137 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.900107 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd8f160a-4f21-4cab-aece-49dd0fe9a3bb" containerName="extract" Jul 09 14:17:29.903193 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.903173 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n7h8m" Jul 09 14:17:29.906484 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.906459 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:17:29.906594 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.906488 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 09 14:17:29.906594 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.906552 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-sm2rc\"" Jul 09 14:17:29.921369 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.921344 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n7h8m"] Jul 09 14:17:29.978498 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.978463 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b171066-6917-48dd-9ee1-050ec0f6cc04-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-n7h8m\" (UID: \"1b171066-6917-48dd-9ee1-050ec0f6cc04\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n7h8m" Jul 09 14:17:29.978667 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:29.978507 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dls6t\" (UniqueName: \"kubernetes.io/projected/1b171066-6917-48dd-9ee1-050ec0f6cc04-kube-api-access-dls6t\") pod \"cert-manager-operator-controller-manager-55c444bf86-n7h8m\" (UID: \"1b171066-6917-48dd-9ee1-050ec0f6cc04\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n7h8m" Jul 09 14:17:30.079362 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:30.079309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b171066-6917-48dd-9ee1-050ec0f6cc04-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-n7h8m\" (UID: \"1b171066-6917-48dd-9ee1-050ec0f6cc04\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n7h8m" Jul 09 14:17:30.079543 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:30.079386 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dls6t\" (UniqueName: \"kubernetes.io/projected/1b171066-6917-48dd-9ee1-050ec0f6cc04-kube-api-access-dls6t\") pod \"cert-manager-operator-controller-manager-55c444bf86-n7h8m\" (UID: \"1b171066-6917-48dd-9ee1-050ec0f6cc04\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n7h8m" Jul 09 14:17:30.079743 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:30.079725 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b171066-6917-48dd-9ee1-050ec0f6cc04-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-n7h8m\" (UID: \"1b171066-6917-48dd-9ee1-050ec0f6cc04\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n7h8m" Jul 09 14:17:30.088686 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:30.088651 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dls6t\" (UniqueName: \"kubernetes.io/projected/1b171066-6917-48dd-9ee1-050ec0f6cc04-kube-api-access-dls6t\") pod \"cert-manager-operator-controller-manager-55c444bf86-n7h8m\" (UID: \"1b171066-6917-48dd-9ee1-050ec0f6cc04\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n7h8m" Jul 09 14:17:30.211654 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:30.211568 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n7h8m" Jul 09 14:17:30.349913 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:30.349882 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n7h8m"] Jul 09 14:17:30.353135 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:17:30.353103 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b171066_6917_48dd_9ee1_050ec0f6cc04.slice/crio-9c24a6f579e2763719b2f3204422d3daf9d4e209e3c76696922123f08dbf1d7a WatchSource:0}: Error finding container 9c24a6f579e2763719b2f3204422d3daf9d4e209e3c76696922123f08dbf1d7a: Status 404 returned error can't find the container with id 9c24a6f579e2763719b2f3204422d3daf9d4e209e3c76696922123f08dbf1d7a Jul 09 14:17:30.803033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:30.802994 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n7h8m" event={"ID":"1b171066-6917-48dd-9ee1-050ec0f6cc04","Type":"ContainerStarted","Data":"9c24a6f579e2763719b2f3204422d3daf9d4e209e3c76696922123f08dbf1d7a"} Jul 09 14:17:32.814729 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:32.814634 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n7h8m" event={"ID":"1b171066-6917-48dd-9ee1-050ec0f6cc04","Type":"ContainerStarted","Data":"cb4f4e2163ac743ce009124f8a770443500246737f2d2bfe8f85e3dd17248616"} Jul 09 14:17:32.839385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:32.839321 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n7h8m" podStartSLOduration=1.825281068 podStartE2EDuration="3.83930004s" totalImagesPullingTime="2.014018972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:17:32.837290399 +0000 UTC m=+172.960429805" watchObservedRunningTime="2026-07-09 14:17:32.83930004 +0000 UTC m=+172.962439426" Jul 09 14:17:34.097257 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.097223 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf"] Jul 09 14:17:34.100936 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.100913 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" Jul 09 14:17:34.104204 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.104171 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tr2j5\"" Jul 09 14:17:34.104346 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.104210 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 14:17:34.104346 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.104222 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 14:17:34.113273 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.113248 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf"] Jul 09 14:17:34.216990 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.216934 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df312be-62fc-40ba-8aeb-c85cf501fed1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf\" (UID: \"6df312be-62fc-40ba-8aeb-c85cf501fed1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" Jul 09 14:17:34.217163 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.217006 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df312be-62fc-40ba-8aeb-c85cf501fed1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf\" (UID: \"6df312be-62fc-40ba-8aeb-c85cf501fed1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" Jul 09 14:17:34.217163 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.217034 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt9d2\" (UniqueName: \"kubernetes.io/projected/6df312be-62fc-40ba-8aeb-c85cf501fed1-kube-api-access-mt9d2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf\" (UID: \"6df312be-62fc-40ba-8aeb-c85cf501fed1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" Jul 09 14:17:34.317618 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.317577 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df312be-62fc-40ba-8aeb-c85cf501fed1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf\" (UID: \"6df312be-62fc-40ba-8aeb-c85cf501fed1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" Jul 09 14:17:34.317618 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.317620 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df312be-62fc-40ba-8aeb-c85cf501fed1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf\" (UID: \"6df312be-62fc-40ba-8aeb-c85cf501fed1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" Jul 09 14:17:34.317825 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.317649 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt9d2\" (UniqueName: \"kubernetes.io/projected/6df312be-62fc-40ba-8aeb-c85cf501fed1-kube-api-access-mt9d2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf\" (UID: \"6df312be-62fc-40ba-8aeb-c85cf501fed1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" Jul 09 14:17:34.318037 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.318017 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df312be-62fc-40ba-8aeb-c85cf501fed1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf\" (UID: \"6df312be-62fc-40ba-8aeb-c85cf501fed1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" Jul 09 14:17:34.318077 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.318047 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df312be-62fc-40ba-8aeb-c85cf501fed1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf\" (UID: \"6df312be-62fc-40ba-8aeb-c85cf501fed1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" Jul 09 14:17:34.330288 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.330252 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt9d2\" (UniqueName: \"kubernetes.io/projected/6df312be-62fc-40ba-8aeb-c85cf501fed1-kube-api-access-mt9d2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf\" (UID: \"6df312be-62fc-40ba-8aeb-c85cf501fed1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" Jul 09 14:17:34.410188 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.410098 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" Jul 09 14:17:34.543125 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.543100 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf"] Jul 09 14:17:34.545431 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:17:34.545405 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df312be_62fc_40ba_8aeb_c85cf501fed1.slice/crio-cc7ecc3154fdaee958c767c97242bb6f97b1cd289c26718c3187f5ee20db8e03 WatchSource:0}: Error finding container cc7ecc3154fdaee958c767c97242bb6f97b1cd289c26718c3187f5ee20db8e03: Status 404 returned error can't find the container with id cc7ecc3154fdaee958c767c97242bb6f97b1cd289c26718c3187f5ee20db8e03 Jul 09 14:17:34.823913 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.823824 2581 generic.go:356] "Generic (PLEG): container finished" podID="6df312be-62fc-40ba-8aeb-c85cf501fed1" containerID="24faeb58ff3879e85a570ac5185345cc1c43ecf10054a48d1f43f70d80e6a064" exitCode=0 Jul 09 14:17:34.824086 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.823928 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" event={"ID":"6df312be-62fc-40ba-8aeb-c85cf501fed1","Type":"ContainerDied","Data":"24faeb58ff3879e85a570ac5185345cc1c43ecf10054a48d1f43f70d80e6a064"} Jul 09 14:17:34.824086 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:34.824009 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" event={"ID":"6df312be-62fc-40ba-8aeb-c85cf501fed1","Type":"ContainerStarted","Data":"cc7ecc3154fdaee958c767c97242bb6f97b1cd289c26718c3187f5ee20db8e03"} Jul 09 14:17:36.749475 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:36.749443 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-drjts"] Jul 09 14:17:36.752715 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:36.752689 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-drjts" Jul 09 14:17:36.756264 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:36.756225 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 14:17:36.757573 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:36.757549 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 14:17:36.757689 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:36.757549 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-kglhp\"" Jul 09 14:17:36.761102 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:36.761015 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-drjts"] Jul 09 14:17:36.841701 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:36.841661 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc95a780-620c-4473-a78f-3e945ac742a3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-drjts\" (UID: \"bc95a780-620c-4473-a78f-3e945ac742a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-drjts" Jul 09 14:17:36.841866 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:36.841732 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js6gs\" (UniqueName: \"kubernetes.io/projected/bc95a780-620c-4473-a78f-3e945ac742a3-kube-api-access-js6gs\") pod \"cert-manager-cainjector-559cdd8c87-drjts\" (UID: \"bc95a780-620c-4473-a78f-3e945ac742a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-drjts" Jul 09 14:17:36.943186 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:36.943138 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc95a780-620c-4473-a78f-3e945ac742a3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-drjts\" (UID: \"bc95a780-620c-4473-a78f-3e945ac742a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-drjts" Jul 09 14:17:36.943385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:36.943332 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js6gs\" (UniqueName: \"kubernetes.io/projected/bc95a780-620c-4473-a78f-3e945ac742a3-kube-api-access-js6gs\") pod \"cert-manager-cainjector-559cdd8c87-drjts\" (UID: \"bc95a780-620c-4473-a78f-3e945ac742a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-drjts" Jul 09 14:17:36.953067 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:36.953034 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc95a780-620c-4473-a78f-3e945ac742a3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-drjts\" (UID: \"bc95a780-620c-4473-a78f-3e945ac742a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-drjts" Jul 09 14:17:36.953303 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:36.953281 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js6gs\" (UniqueName: \"kubernetes.io/projected/bc95a780-620c-4473-a78f-3e945ac742a3-kube-api-access-js6gs\") pod \"cert-manager-cainjector-559cdd8c87-drjts\" (UID: \"bc95a780-620c-4473-a78f-3e945ac742a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-drjts" Jul 09 14:17:37.075316 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:37.075236 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-drjts" Jul 09 14:17:37.795427 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:37.795400 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-drjts"] Jul 09 14:17:37.798199 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:17:37.798176 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc95a780_620c_4473_a78f_3e945ac742a3.slice/crio-8caa64ec453733331479b79aac172613a736e9c8b54d6c87cd8e98851f55e78e WatchSource:0}: Error finding container 8caa64ec453733331479b79aac172613a736e9c8b54d6c87cd8e98851f55e78e: Status 404 returned error can't find the container with id 8caa64ec453733331479b79aac172613a736e9c8b54d6c87cd8e98851f55e78e Jul 09 14:17:37.839944 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:37.839892 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" event={"ID":"6df312be-62fc-40ba-8aeb-c85cf501fed1","Type":"ContainerStarted","Data":"cfebe56910cde75fcafce0e1dff0e313faf934659d5ea01420f50a9e0af6be34"} Jul 09 14:17:37.841251 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:37.841221 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-drjts" event={"ID":"bc95a780-620c-4473-a78f-3e945ac742a3","Type":"ContainerStarted","Data":"8caa64ec453733331479b79aac172613a736e9c8b54d6c87cd8e98851f55e78e"} Jul 09 14:17:38.848096 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:38.848059 2581 generic.go:356] "Generic (PLEG): container finished" podID="6df312be-62fc-40ba-8aeb-c85cf501fed1" containerID="cfebe56910cde75fcafce0e1dff0e313faf934659d5ea01420f50a9e0af6be34" exitCode=0 Jul 09 14:17:38.848544 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:38.848137 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" event={"ID":"6df312be-62fc-40ba-8aeb-c85cf501fed1","Type":"ContainerDied","Data":"cfebe56910cde75fcafce0e1dff0e313faf934659d5ea01420f50a9e0af6be34"} Jul 09 14:17:39.858043 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:39.858004 2581 generic.go:356] "Generic (PLEG): container finished" podID="6df312be-62fc-40ba-8aeb-c85cf501fed1" containerID="fd03f83c856a0a8a4e9b73cdfe9b0a01e8fd1149ff46f53b65287ae3dddd64b5" exitCode=0 Jul 09 14:17:39.858503 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:39.858098 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" event={"ID":"6df312be-62fc-40ba-8aeb-c85cf501fed1","Type":"ContainerDied","Data":"fd03f83c856a0a8a4e9b73cdfe9b0a01e8fd1149ff46f53b65287ae3dddd64b5"} Jul 09 14:17:41.643530 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.643504 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" Jul 09 14:17:41.789043 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.789012 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df312be-62fc-40ba-8aeb-c85cf501fed1-util\") pod \"6df312be-62fc-40ba-8aeb-c85cf501fed1\" (UID: \"6df312be-62fc-40ba-8aeb-c85cf501fed1\") " Jul 09 14:17:41.789189 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.789120 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df312be-62fc-40ba-8aeb-c85cf501fed1-bundle\") pod \"6df312be-62fc-40ba-8aeb-c85cf501fed1\" (UID: \"6df312be-62fc-40ba-8aeb-c85cf501fed1\") " Jul 09 14:17:41.789189 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.789154 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt9d2\" (UniqueName: \"kubernetes.io/projected/6df312be-62fc-40ba-8aeb-c85cf501fed1-kube-api-access-mt9d2\") pod \"6df312be-62fc-40ba-8aeb-c85cf501fed1\" (UID: \"6df312be-62fc-40ba-8aeb-c85cf501fed1\") " Jul 09 14:17:41.789475 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.789451 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6df312be-62fc-40ba-8aeb-c85cf501fed1-bundle" (OuterVolumeSpecName: "bundle") pod "6df312be-62fc-40ba-8aeb-c85cf501fed1" (UID: "6df312be-62fc-40ba-8aeb-c85cf501fed1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:17:41.791583 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.791548 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6df312be-62fc-40ba-8aeb-c85cf501fed1-kube-api-access-mt9d2" (OuterVolumeSpecName: "kube-api-access-mt9d2") pod "6df312be-62fc-40ba-8aeb-c85cf501fed1" (UID: "6df312be-62fc-40ba-8aeb-c85cf501fed1"). InnerVolumeSpecName "kube-api-access-mt9d2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:17:41.794021 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.793998 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6df312be-62fc-40ba-8aeb-c85cf501fed1-util" (OuterVolumeSpecName: "util") pod "6df312be-62fc-40ba-8aeb-c85cf501fed1" (UID: "6df312be-62fc-40ba-8aeb-c85cf501fed1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:17:41.869780 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.869752 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" Jul 09 14:17:41.869780 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.869772 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8sbdf" event={"ID":"6df312be-62fc-40ba-8aeb-c85cf501fed1","Type":"ContainerDied","Data":"cc7ecc3154fdaee958c767c97242bb6f97b1cd289c26718c3187f5ee20db8e03"} Jul 09 14:17:41.870024 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.869827 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc7ecc3154fdaee958c767c97242bb6f97b1cd289c26718c3187f5ee20db8e03" Jul 09 14:17:41.871558 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.871522 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-drjts" event={"ID":"bc95a780-620c-4473-a78f-3e945ac742a3","Type":"ContainerStarted","Data":"363a7be0419153f1bc1d6a2543d952009b9e0f4b85afaa6f9d3010cfdf502cfd"} Jul 09 14:17:41.889074 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.889031 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-drjts" podStartSLOduration=1.999758972 podStartE2EDuration="5.889017699s" totalImagesPullingTime="3.889258727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:17:41.887820241 +0000 UTC m=+182.010959642" watchObservedRunningTime="2026-07-09 14:17:41.889017699 +0000 UTC m=+182.012157081" Jul 09 14:17:41.889902 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.889879 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df312be-62fc-40ba-8aeb-c85cf501fed1-util\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:17:41.889902 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.889902 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df312be-62fc-40ba-8aeb-c85cf501fed1-bundle\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:17:41.890042 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:41.889911 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt9d2\" (UniqueName: \"kubernetes.io/projected/6df312be-62fc-40ba-8aeb-c85cf501fed1-kube-api-access-mt9d2\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:17:51.885015 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.884978 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkd5n"] Jul 09 14:17:51.885613 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.885413 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6df312be-62fc-40ba-8aeb-c85cf501fed1" containerName="extract" Jul 09 14:17:51.885613 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.885433 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6df312be-62fc-40ba-8aeb-c85cf501fed1" containerName="extract" Jul 09 14:17:51.885613 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.885457 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6df312be-62fc-40ba-8aeb-c85cf501fed1" containerName="util" Jul 09 14:17:51.885613 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.885466 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6df312be-62fc-40ba-8aeb-c85cf501fed1" containerName="util" Jul 09 14:17:51.885613 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.885490 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6df312be-62fc-40ba-8aeb-c85cf501fed1" containerName="pull" Jul 09 14:17:51.885613 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.885499 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6df312be-62fc-40ba-8aeb-c85cf501fed1" containerName="pull" Jul 09 14:17:51.885613 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.885570 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6df312be-62fc-40ba-8aeb-c85cf501fed1" containerName="extract" Jul 09 14:17:51.889487 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.889468 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkd5n" Jul 09 14:17:51.892456 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.892435 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:17:51.892589 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.892501 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 14:17:51.894136 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.894114 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-4294s\"" Jul 09 14:17:51.896100 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.896078 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkd5n"] Jul 09 14:17:51.980412 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.980355 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rkd5n\" (UID: \"1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkd5n" Jul 09 14:17:51.980592 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:51.980498 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv9r6\" (UniqueName: \"kubernetes.io/projected/1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc-kube-api-access-fv9r6\") pod \"openshift-lws-operator-fd8ccff4c-rkd5n\" (UID: \"1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkd5n" Jul 09 14:17:52.081733 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:52.081693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rkd5n\" (UID: \"1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkd5n" Jul 09 14:17:52.081913 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:52.081772 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv9r6\" (UniqueName: \"kubernetes.io/projected/1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc-kube-api-access-fv9r6\") pod \"openshift-lws-operator-fd8ccff4c-rkd5n\" (UID: \"1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkd5n" Jul 09 14:17:52.082099 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:52.082078 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rkd5n\" (UID: \"1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkd5n" Jul 09 14:17:52.091278 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:52.091253 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv9r6\" (UniqueName: \"kubernetes.io/projected/1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc-kube-api-access-fv9r6\") pod \"openshift-lws-operator-fd8ccff4c-rkd5n\" (UID: \"1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkd5n" Jul 09 14:17:52.200374 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:52.200281 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkd5n" Jul 09 14:17:52.335214 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:52.335186 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkd5n"] Jul 09 14:17:52.338047 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:17:52.338018 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1030ebb6_047f_4a4a_89b4_ea08fc0fb8fc.slice/crio-71f146b0fb397a9faed1726d1fdd6acea50d15adf716c1a1112ff72579437d18 WatchSource:0}: Error finding container 71f146b0fb397a9faed1726d1fdd6acea50d15adf716c1a1112ff72579437d18: Status 404 returned error can't find the container with id 71f146b0fb397a9faed1726d1fdd6acea50d15adf716c1a1112ff72579437d18 Jul 09 14:17:52.919839 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:17:52.919803 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkd5n" event={"ID":"1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc","Type":"ContainerStarted","Data":"71f146b0fb397a9faed1726d1fdd6acea50d15adf716c1a1112ff72579437d18"} Jul 09 14:18:00.956996 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:00.956959 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkd5n" event={"ID":"1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc","Type":"ContainerStarted","Data":"092a58ee97523ea4eb3f135fe6a03de78ff77c8218fd3791c0e3c20008ebe058"} Jul 09 14:18:00.976240 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:00.976188 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rkd5n" podStartSLOduration=2.117011352 podStartE2EDuration="9.97617174s" totalImagesPullingTime="7.859160388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:18:00.974981408 +0000 UTC m=+201.098120814" watchObservedRunningTime="2026-07-09 14:18:00.97617174 +0000 UTC m=+201.099311122" Jul 09 14:18:04.746435 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.746392 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w"] Jul 09 14:18:04.751230 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.751206 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" Jul 09 14:18:04.754166 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.754144 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 14:18:04.755443 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.755425 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tr2j5\"" Jul 09 14:18:04.755443 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.755439 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 14:18:04.758566 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.758534 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w"] Jul 09 14:18:04.776336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.776309 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a5e4085-77c8-441a-9afc-b4bf274a293c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w\" (UID: \"0a5e4085-77c8-441a-9afc-b4bf274a293c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" Jul 09 14:18:04.776504 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.776350 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2l9j\" (UniqueName: \"kubernetes.io/projected/0a5e4085-77c8-441a-9afc-b4bf274a293c-kube-api-access-c2l9j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w\" (UID: \"0a5e4085-77c8-441a-9afc-b4bf274a293c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" Jul 09 14:18:04.776504 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.776403 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a5e4085-77c8-441a-9afc-b4bf274a293c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w\" (UID: \"0a5e4085-77c8-441a-9afc-b4bf274a293c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" Jul 09 14:18:04.877815 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.877779 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a5e4085-77c8-441a-9afc-b4bf274a293c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w\" (UID: \"0a5e4085-77c8-441a-9afc-b4bf274a293c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" Jul 09 14:18:04.878061 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.877854 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a5e4085-77c8-441a-9afc-b4bf274a293c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w\" (UID: \"0a5e4085-77c8-441a-9afc-b4bf274a293c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" Jul 09 14:18:04.878061 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.877891 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2l9j\" (UniqueName: \"kubernetes.io/projected/0a5e4085-77c8-441a-9afc-b4bf274a293c-kube-api-access-c2l9j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w\" (UID: \"0a5e4085-77c8-441a-9afc-b4bf274a293c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" Jul 09 14:18:04.878217 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.878191 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a5e4085-77c8-441a-9afc-b4bf274a293c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w\" (UID: \"0a5e4085-77c8-441a-9afc-b4bf274a293c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" Jul 09 14:18:04.878275 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.878215 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a5e4085-77c8-441a-9afc-b4bf274a293c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w\" (UID: \"0a5e4085-77c8-441a-9afc-b4bf274a293c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" Jul 09 14:18:04.894910 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:04.894872 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2l9j\" (UniqueName: \"kubernetes.io/projected/0a5e4085-77c8-441a-9afc-b4bf274a293c-kube-api-access-c2l9j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w\" (UID: \"0a5e4085-77c8-441a-9afc-b4bf274a293c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" Jul 09 14:18:05.060453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:05.060356 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" Jul 09 14:18:05.210860 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:05.210836 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w"] Jul 09 14:18:05.212425 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:18:05.212398 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a5e4085_77c8_441a_9afc_b4bf274a293c.slice/crio-9d4fefe9be958f3045bf19eef4db2a0c2297a7d76873fe9c8e5a05c213924356 WatchSource:0}: Error finding container 9d4fefe9be958f3045bf19eef4db2a0c2297a7d76873fe9c8e5a05c213924356: Status 404 returned error can't find the container with id 9d4fefe9be958f3045bf19eef4db2a0c2297a7d76873fe9c8e5a05c213924356 Jul 09 14:18:05.980638 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:05.980605 2581 generic.go:356] "Generic (PLEG): container finished" podID="0a5e4085-77c8-441a-9afc-b4bf274a293c" containerID="dac5d6793c347a3c87432c671675c9341db9fcbacdb51a3d8551e1e5af3c608b" exitCode=0 Jul 09 14:18:05.981025 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:05.980683 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" event={"ID":"0a5e4085-77c8-441a-9afc-b4bf274a293c","Type":"ContainerDied","Data":"dac5d6793c347a3c87432c671675c9341db9fcbacdb51a3d8551e1e5af3c608b"} Jul 09 14:18:05.981025 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:05.980727 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" event={"ID":"0a5e4085-77c8-441a-9afc-b4bf274a293c","Type":"ContainerStarted","Data":"9d4fefe9be958f3045bf19eef4db2a0c2297a7d76873fe9c8e5a05c213924356"} Jul 09 14:18:06.986575 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:06.986536 2581 generic.go:356] "Generic (PLEG): container finished" podID="0a5e4085-77c8-441a-9afc-b4bf274a293c" containerID="70da72eaecab9fdadeac8b4efde35ad11f1e8d4392e8f9367e6e239ea921322b" exitCode=0 Jul 09 14:18:06.986935 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:06.986619 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" event={"ID":"0a5e4085-77c8-441a-9afc-b4bf274a293c","Type":"ContainerDied","Data":"70da72eaecab9fdadeac8b4efde35ad11f1e8d4392e8f9367e6e239ea921322b"} Jul 09 14:18:07.723969 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.723924 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs"] Jul 09 14:18:07.727197 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.727180 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:07.731862 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.731838 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 14:18:07.732029 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.731889 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 14:18:07.732029 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.731918 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 14:18:07.732029 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.731945 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8mjl7\"" Jul 09 14:18:07.740341 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.740319 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs"] Jul 09 14:18:07.804811 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.804783 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4997f2ce-3bc8-4baf-84f9-82728106d0cf-cert\") pod \"lws-controller-manager-5578c64b55-q78qs\" (UID: \"4997f2ce-3bc8-4baf-84f9-82728106d0cf\") " pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:07.805002 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.804818 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4997f2ce-3bc8-4baf-84f9-82728106d0cf-manager-config\") pod \"lws-controller-manager-5578c64b55-q78qs\" (UID: \"4997f2ce-3bc8-4baf-84f9-82728106d0cf\") " pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:07.805002 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.804847 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bm97\" (UniqueName: \"kubernetes.io/projected/4997f2ce-3bc8-4baf-84f9-82728106d0cf-kube-api-access-5bm97\") pod \"lws-controller-manager-5578c64b55-q78qs\" (UID: \"4997f2ce-3bc8-4baf-84f9-82728106d0cf\") " pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:07.805002 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.804941 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4997f2ce-3bc8-4baf-84f9-82728106d0cf-metrics-cert\") pod \"lws-controller-manager-5578c64b55-q78qs\" (UID: \"4997f2ce-3bc8-4baf-84f9-82728106d0cf\") " pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:07.906188 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.906145 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4997f2ce-3bc8-4baf-84f9-82728106d0cf-manager-config\") pod \"lws-controller-manager-5578c64b55-q78qs\" (UID: \"4997f2ce-3bc8-4baf-84f9-82728106d0cf\") " pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:07.906393 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.906195 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bm97\" (UniqueName: \"kubernetes.io/projected/4997f2ce-3bc8-4baf-84f9-82728106d0cf-kube-api-access-5bm97\") pod \"lws-controller-manager-5578c64b55-q78qs\" (UID: \"4997f2ce-3bc8-4baf-84f9-82728106d0cf\") " pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:07.906393 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.906264 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4997f2ce-3bc8-4baf-84f9-82728106d0cf-metrics-cert\") pod \"lws-controller-manager-5578c64b55-q78qs\" (UID: \"4997f2ce-3bc8-4baf-84f9-82728106d0cf\") " pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:07.906393 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.906334 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4997f2ce-3bc8-4baf-84f9-82728106d0cf-cert\") pod \"lws-controller-manager-5578c64b55-q78qs\" (UID: \"4997f2ce-3bc8-4baf-84f9-82728106d0cf\") " pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:07.906926 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.906897 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4997f2ce-3bc8-4baf-84f9-82728106d0cf-manager-config\") pod \"lws-controller-manager-5578c64b55-q78qs\" (UID: \"4997f2ce-3bc8-4baf-84f9-82728106d0cf\") " pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:07.908757 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.908729 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4997f2ce-3bc8-4baf-84f9-82728106d0cf-metrics-cert\") pod \"lws-controller-manager-5578c64b55-q78qs\" (UID: \"4997f2ce-3bc8-4baf-84f9-82728106d0cf\") " pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:07.908851 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.908772 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4997f2ce-3bc8-4baf-84f9-82728106d0cf-cert\") pod \"lws-controller-manager-5578c64b55-q78qs\" (UID: \"4997f2ce-3bc8-4baf-84f9-82728106d0cf\") " pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:07.916043 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.916023 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bm97\" (UniqueName: \"kubernetes.io/projected/4997f2ce-3bc8-4baf-84f9-82728106d0cf-kube-api-access-5bm97\") pod \"lws-controller-manager-5578c64b55-q78qs\" (UID: \"4997f2ce-3bc8-4baf-84f9-82728106d0cf\") " pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:07.993328 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.993230 2581 generic.go:356] "Generic (PLEG): container finished" podID="0a5e4085-77c8-441a-9afc-b4bf274a293c" containerID="70c729dc062c2c879fdbc4924e4b8d0a85cbfbbd30cd655e409da6e749c8f206" exitCode=0 Jul 09 14:18:07.993328 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:07.993305 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" event={"ID":"0a5e4085-77c8-441a-9afc-b4bf274a293c","Type":"ContainerDied","Data":"70c729dc062c2c879fdbc4924e4b8d0a85cbfbbd30cd655e409da6e749c8f206"} Jul 09 14:18:08.041100 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:08.041065 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:08.189364 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:08.189339 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs"] Jul 09 14:18:08.191595 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:18:08.191567 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4997f2ce_3bc8_4baf_84f9_82728106d0cf.slice/crio-4e0cfecf34873fa19c3a0241a6092d3152ea9b1086cca28cae16b2cc3e24664b WatchSource:0}: Error finding container 4e0cfecf34873fa19c3a0241a6092d3152ea9b1086cca28cae16b2cc3e24664b: Status 404 returned error can't find the container with id 4e0cfecf34873fa19c3a0241a6092d3152ea9b1086cca28cae16b2cc3e24664b Jul 09 14:18:08.998523 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:08.998488 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" event={"ID":"4997f2ce-3bc8-4baf-84f9-82728106d0cf","Type":"ContainerStarted","Data":"4e0cfecf34873fa19c3a0241a6092d3152ea9b1086cca28cae16b2cc3e24664b"} Jul 09 14:18:09.125306 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:09.125280 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" Jul 09 14:18:09.218227 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:09.218194 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a5e4085-77c8-441a-9afc-b4bf274a293c-util\") pod \"0a5e4085-77c8-441a-9afc-b4bf274a293c\" (UID: \"0a5e4085-77c8-441a-9afc-b4bf274a293c\") " Jul 09 14:18:09.218427 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:09.218260 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2l9j\" (UniqueName: \"kubernetes.io/projected/0a5e4085-77c8-441a-9afc-b4bf274a293c-kube-api-access-c2l9j\") pod \"0a5e4085-77c8-441a-9afc-b4bf274a293c\" (UID: \"0a5e4085-77c8-441a-9afc-b4bf274a293c\") " Jul 09 14:18:09.218427 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:09.218346 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a5e4085-77c8-441a-9afc-b4bf274a293c-bundle\") pod \"0a5e4085-77c8-441a-9afc-b4bf274a293c\" (UID: \"0a5e4085-77c8-441a-9afc-b4bf274a293c\") " Jul 09 14:18:09.218974 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:09.218935 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5e4085-77c8-441a-9afc-b4bf274a293c-bundle" (OuterVolumeSpecName: "bundle") pod "0a5e4085-77c8-441a-9afc-b4bf274a293c" (UID: "0a5e4085-77c8-441a-9afc-b4bf274a293c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:18:09.220415 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:09.220387 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a5e4085-77c8-441a-9afc-b4bf274a293c-kube-api-access-c2l9j" (OuterVolumeSpecName: "kube-api-access-c2l9j") pod "0a5e4085-77c8-441a-9afc-b4bf274a293c" (UID: "0a5e4085-77c8-441a-9afc-b4bf274a293c"). InnerVolumeSpecName "kube-api-access-c2l9j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:18:09.223924 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:09.223897 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5e4085-77c8-441a-9afc-b4bf274a293c-util" (OuterVolumeSpecName: "util") pod "0a5e4085-77c8-441a-9afc-b4bf274a293c" (UID: "0a5e4085-77c8-441a-9afc-b4bf274a293c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:18:09.319153 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:09.319064 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a5e4085-77c8-441a-9afc-b4bf274a293c-util\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:18:09.319153 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:09.319096 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2l9j\" (UniqueName: \"kubernetes.io/projected/0a5e4085-77c8-441a-9afc-b4bf274a293c-kube-api-access-c2l9j\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:18:09.319153 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:09.319113 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a5e4085-77c8-441a-9afc-b4bf274a293c-bundle\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:18:10.006326 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:10.006297 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" Jul 09 14:18:10.006791 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:10.006295 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzhg4w" event={"ID":"0a5e4085-77c8-441a-9afc-b4bf274a293c","Type":"ContainerDied","Data":"9d4fefe9be958f3045bf19eef4db2a0c2297a7d76873fe9c8e5a05c213924356"} Jul 09 14:18:10.006791 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:10.006446 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d4fefe9be958f3045bf19eef4db2a0c2297a7d76873fe9c8e5a05c213924356" Jul 09 14:18:11.012203 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:11.012167 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" event={"ID":"4997f2ce-3bc8-4baf-84f9-82728106d0cf","Type":"ContainerStarted","Data":"2c0c6a3de62c37810213b3b569075eba414bdf714ad5b9bec4187c91f5fcaae6"} Jul 09 14:18:11.012601 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:11.012260 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:11.033334 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:11.033286 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" podStartSLOduration=2.14389388 podStartE2EDuration="4.033272139s" totalImagesPullingTime="1.889378259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:18:11.032026295 +0000 UTC m=+211.155165678" watchObservedRunningTime="2026-07-09 14:18:11.033272139 +0000 UTC m=+211.156411522" Jul 09 14:18:16.489834 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.489800 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl"] Jul 09 14:18:16.490241 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.490114 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a5e4085-77c8-441a-9afc-b4bf274a293c" containerName="extract" Jul 09 14:18:16.490241 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.490127 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a5e4085-77c8-441a-9afc-b4bf274a293c" containerName="extract" Jul 09 14:18:16.490241 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.490142 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a5e4085-77c8-441a-9afc-b4bf274a293c" containerName="util" Jul 09 14:18:16.490241 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.490148 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a5e4085-77c8-441a-9afc-b4bf274a293c" containerName="util" Jul 09 14:18:16.490241 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.490162 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a5e4085-77c8-441a-9afc-b4bf274a293c" containerName="pull" Jul 09 14:18:16.490241 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.490167 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a5e4085-77c8-441a-9afc-b4bf274a293c" containerName="pull" Jul 09 14:18:16.490241 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.490217 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a5e4085-77c8-441a-9afc-b4bf274a293c" containerName="extract" Jul 09 14:18:16.492723 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.492705 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.495502 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.495470 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 14:18:16.495643 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.495513 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 14:18:16.495802 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.495785 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 14:18:16.496222 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.496203 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 14:18:16.496396 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.496380 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qf47d\"" Jul 09 14:18:16.512815 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.512788 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl"] Jul 09 14:18:16.578877 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.578837 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae8d4c12-0ee3-444d-ba0e-9f749336b750-tmp\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.579107 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.578892 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae8d4c12-0ee3-444d-ba0e-9f749336b750-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.579107 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.579001 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae8d4c12-0ee3-444d-ba0e-9f749336b750-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.579107 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.579069 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae8d4c12-0ee3-444d-ba0e-9f749336b750-manifests\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.579249 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.579130 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzvd9\" (UniqueName: \"kubernetes.io/projected/ae8d4c12-0ee3-444d-ba0e-9f749336b750-kube-api-access-bzvd9\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.680101 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.680066 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae8d4c12-0ee3-444d-ba0e-9f749336b750-tmp\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.680274 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.680110 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae8d4c12-0ee3-444d-ba0e-9f749336b750-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.680274 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.680147 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae8d4c12-0ee3-444d-ba0e-9f749336b750-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.680274 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.680183 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae8d4c12-0ee3-444d-ba0e-9f749336b750-manifests\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.680274 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.680220 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzvd9\" (UniqueName: \"kubernetes.io/projected/ae8d4c12-0ee3-444d-ba0e-9f749336b750-kube-api-access-bzvd9\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.680630 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.680603 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae8d4c12-0ee3-444d-ba0e-9f749336b750-manifests\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.682374 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.682347 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae8d4c12-0ee3-444d-ba0e-9f749336b750-tmp\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.682589 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.682572 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae8d4c12-0ee3-444d-ba0e-9f749336b750-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.682632 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.682602 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae8d4c12-0ee3-444d-ba0e-9f749336b750-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.691835 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.691810 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzvd9\" (UniqueName: \"kubernetes.io/projected/ae8d4c12-0ee3-444d-ba0e-9f749336b750-kube-api-access-bzvd9\") pod \"opendatahub-operator-controller-manager-5f8568cbd5-8nctl\" (UID: \"ae8d4c12-0ee3-444d-ba0e-9f749336b750\") " pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.803063 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.802974 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:16.940429 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:16.940396 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl"] Jul 09 14:18:16.944370 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:18:16.944336 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae8d4c12_0ee3_444d_ba0e_9f749336b750.slice/crio-78d0dd54c1bdb2d49379b0371894c1eb05e79fe480f15f22f9134ac2a7284624 WatchSource:0}: Error finding container 78d0dd54c1bdb2d49379b0371894c1eb05e79fe480f15f22f9134ac2a7284624: Status 404 returned error can't find the container with id 78d0dd54c1bdb2d49379b0371894c1eb05e79fe480f15f22f9134ac2a7284624 Jul 09 14:18:17.038610 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:17.038566 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" event={"ID":"ae8d4c12-0ee3-444d-ba0e-9f749336b750","Type":"ContainerStarted","Data":"78d0dd54c1bdb2d49379b0371894c1eb05e79fe480f15f22f9134ac2a7284624"} Jul 09 14:18:20.055811 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:20.055769 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" event={"ID":"ae8d4c12-0ee3-444d-ba0e-9f749336b750","Type":"ContainerStarted","Data":"9fbd4ca9b9a3d14b6adf900b3a8e7ad88a1dbe3999478bde18536c524767afc9"} Jul 09 14:18:21.061967 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:21.061918 2581 generic.go:356] "Generic (PLEG): container finished" podID="ae8d4c12-0ee3-444d-ba0e-9f749336b750" containerID="9fbd4ca9b9a3d14b6adf900b3a8e7ad88a1dbe3999478bde18536c524767afc9" exitCode=0 Jul 09 14:18:21.062338 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:21.062005 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" event={"ID":"ae8d4c12-0ee3-444d-ba0e-9f749336b750","Type":"ContainerDied","Data":"9fbd4ca9b9a3d14b6adf900b3a8e7ad88a1dbe3999478bde18536c524767afc9"} Jul 09 14:18:22.019349 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:22.019311 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5578c64b55-q78qs" Jul 09 14:18:22.070581 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:22.070541 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" event={"ID":"ae8d4c12-0ee3-444d-ba0e-9f749336b750","Type":"ContainerStarted","Data":"fb24b5ebf6202825b844e6e39106520a819075d9565d5d3db800fece998f7d03"} Jul 09 14:18:22.071109 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:22.070636 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:22.096257 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:22.096194 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" podStartSLOduration=1.688534464 podStartE2EDuration="6.096171313s" totalImagesPullingTime="3.397319218s" totalInitContainerRuntime="1.010317631s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:18:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:18:22.094378293 +0000 UTC m=+222.217517714" watchObservedRunningTime="2026-07-09 14:18:22.096171313 +0000 UTC m=+222.219310696" Jul 09 14:18:33.077941 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:33.077911 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5f8568cbd5-8nctl" Jul 09 14:18:44.254201 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.254164 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96"] Jul 09 14:18:44.261080 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.261057 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.264144 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.264112 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 14:18:44.264290 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.264188 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 14:18:44.264290 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.264201 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-m9lsk\"" Jul 09 14:18:44.264290 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.264197 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 14:18:44.264290 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.264207 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 14:18:44.264524 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.264510 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 14:18:44.264565 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.264535 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 14:18:44.269760 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.269736 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96"] Jul 09 14:18:44.337674 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.337633 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.337853 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.337681 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.337853 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.337726 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.337853 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.337753 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.337853 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.337825 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.338038 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.337868 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.338038 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.337888 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mk7n\" (UniqueName: \"kubernetes.io/projected/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-kube-api-access-8mk7n\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.438931 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.438894 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.438931 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.438928 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.439190 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.438985 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.439190 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.439012 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.439190 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.439048 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mk7n\" (UniqueName: \"kubernetes.io/projected/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-kube-api-access-8mk7n\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.439190 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.439139 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.439190 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.439164 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.439710 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.439674 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.441429 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.441404 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.441607 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.441583 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.441717 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.441665 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.441895 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.441873 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.448781 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.448750 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.448996 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.448974 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mk7n\" (UniqueName: \"kubernetes.io/projected/c40073d1-9c67-4907-b4d8-132c7b6dc4cc-kube-api-access-8mk7n\") pod \"istiod-openshift-gateway-94bb8fbfd-b5v96\" (UID: \"c40073d1-9c67-4907-b4d8-132c7b6dc4cc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.572131 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.572030 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:44.722363 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.722329 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96"] Jul 09 14:18:44.724484 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:18:44.724448 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc40073d1_9c67_4907_b4d8_132c7b6dc4cc.slice/crio-5af8a5bf35f1d511c8e1c1aed71459873c682aff8663db492cb5f812c7554239 WatchSource:0}: Error finding container 5af8a5bf35f1d511c8e1c1aed71459873c682aff8663db492cb5f812c7554239: Status 404 returned error can't find the container with id 5af8a5bf35f1d511c8e1c1aed71459873c682aff8663db492cb5f812c7554239 Jul 09 14:18:44.943313 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.940880 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps"] Jul 09 14:18:44.947767 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.947739 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:44.951122 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.951091 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-gjtbb\"" Jul 09 14:18:44.958053 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:44.958026 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps"] Jul 09 14:18:45.046185 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.046144 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.046185 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.046182 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b33aa88-9054-49e3-ba4d-14d98fa93454-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.046440 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.046207 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.046440 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.046233 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b33aa88-9054-49e3-ba4d-14d98fa93454-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.046440 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.046322 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.046440 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.046404 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.046440 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.046429 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b33aa88-9054-49e3-ba4d-14d98fa93454-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.046678 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.046478 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.046678 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.046575 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6767\" (UniqueName: \"kubernetes.io/projected/9b33aa88-9054-49e3-ba4d-14d98fa93454-kube-api-access-f6767\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.147323 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.147280 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6767\" (UniqueName: \"kubernetes.io/projected/9b33aa88-9054-49e3-ba4d-14d98fa93454-kube-api-access-f6767\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.147517 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.147350 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.147517 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.147382 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b33aa88-9054-49e3-ba4d-14d98fa93454-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.147517 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.147411 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.147517 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.147440 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b33aa88-9054-49e3-ba4d-14d98fa93454-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.147517 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.147473 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.147781 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.147531 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.147781 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.147559 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b33aa88-9054-49e3-ba4d-14d98fa93454-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.147781 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.147598 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.147933 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.147917 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.148353 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.148323 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.148484 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.148435 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.148484 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.148449 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.148640 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.148615 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b33aa88-9054-49e3-ba4d-14d98fa93454-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.149968 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.149920 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b33aa88-9054-49e3-ba4d-14d98fa93454-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.150159 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.150141 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b33aa88-9054-49e3-ba4d-14d98fa93454-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.165337 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.165312 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6767\" (UniqueName: \"kubernetes.io/projected/9b33aa88-9054-49e3-ba4d-14d98fa93454-kube-api-access-f6767\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.165930 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.165906 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b33aa88-9054-49e3-ba4d-14d98fa93454-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rvps\" (UID: \"9b33aa88-9054-49e3-ba4d-14d98fa93454\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.176829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.176792 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" event={"ID":"c40073d1-9c67-4907-b4d8-132c7b6dc4cc","Type":"ContainerStarted","Data":"5af8a5bf35f1d511c8e1c1aed71459873c682aff8663db492cb5f812c7554239"} Jul 09 14:18:45.258901 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.258814 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:45.418855 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:45.418827 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps"] Jul 09 14:18:45.419697 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:18:45.419674 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b33aa88_9054_49e3_ba4d_14d98fa93454.slice/crio-1052557c0e32f88a1c14229ff101379eca0b1679dbd0dd67a5b510d7199e326b WatchSource:0}: Error finding container 1052557c0e32f88a1c14229ff101379eca0b1679dbd0dd67a5b510d7199e326b: Status 404 returned error can't find the container with id 1052557c0e32f88a1c14229ff101379eca0b1679dbd0dd67a5b510d7199e326b Jul 09 14:18:46.183409 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:46.183370 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" event={"ID":"9b33aa88-9054-49e3-ba4d-14d98fa93454","Type":"ContainerStarted","Data":"1052557c0e32f88a1c14229ff101379eca0b1679dbd0dd67a5b510d7199e326b"} Jul 09 14:18:49.046356 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:49.046304 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:18:49.046645 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:49.046390 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:18:49.049031 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:49.049002 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:18:49.049108 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:49.049068 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:18:49.049108 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:49.049101 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:18:49.202885 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:49.202837 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" event={"ID":"c40073d1-9c67-4907-b4d8-132c7b6dc4cc","Type":"ContainerStarted","Data":"7bc5bd581d3e6200d83284653edcae18aef18b7450406861b2bb68984b5a29ea"} Jul 09 14:18:49.203081 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:49.202995 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:18:49.204814 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:49.204786 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" event={"ID":"9b33aa88-9054-49e3-ba4d-14d98fa93454","Type":"ContainerStarted","Data":"e8441c4678a1a90f550109aeaa9e4bc3aefdc22e50ec059e55f95a40dde96227"} Jul 09 14:18:49.259206 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:49.259171 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:49.267913 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:49.267855 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" podStartSLOduration=0.948363603 podStartE2EDuration="5.267832418s" totalImagesPullingTime="4.319468815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:18:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:18:49.238255122 +0000 UTC m=+249.361394506" watchObservedRunningTime="2026-07-09 14:18:49.267832418 +0000 UTC m=+249.390971803" Jul 09 14:18:49.268213 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:49.268184 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" podStartSLOduration=1.64120761 podStartE2EDuration="5.268175078s" totalImagesPullingTime="3.626967468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:18:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:18:49.266337838 +0000 UTC m=+249.389477253" watchObservedRunningTime="2026-07-09 14:18:49.268175078 +0000 UTC m=+249.391314463" Jul 09 14:18:50.211530 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:50.211497 2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-b5v96 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 14:18:50.211903 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:50.211558 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" podUID="c40073d1-9c67-4907-b4d8-132c7b6dc4cc" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:18:50.264334 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:50.264306 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:51.214465 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:51.214432 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:51.215609 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:51.215588 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rvps" Jul 09 14:18:53.210617 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:18:53.210584 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b5v96" Jul 09 14:19:40.626769 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:40.626738 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:19:40.629015 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:40.628992 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:19:40.633986 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:40.633944 2581 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 14:19:52.819486 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:52.819448 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8"] Jul 09 14:19:52.821810 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:52.821787 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" Jul 09 14:19:52.824624 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:52.824604 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 14:19:52.824722 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:52.824603 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gcv6t\"" Jul 09 14:19:52.826164 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:52.826150 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 14:19:52.832235 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:52.832212 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8"] Jul 09 14:19:52.953512 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:52.953477 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4zwt\" (UniqueName: \"kubernetes.io/projected/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-kube-api-access-c4zwt\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8\" (UID: \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" Jul 09 14:19:52.953711 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:52.953521 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8\" (UID: \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" Jul 09 14:19:52.953711 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:52.953583 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8\" (UID: \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" Jul 09 14:19:53.054609 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.054568 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4zwt\" (UniqueName: \"kubernetes.io/projected/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-kube-api-access-c4zwt\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8\" (UID: \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" Jul 09 14:19:53.054802 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.054620 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8\" (UID: \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" Jul 09 14:19:53.054802 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.054662 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8\" (UID: \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" Jul 09 14:19:53.055097 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.055072 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8\" (UID: \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" Jul 09 14:19:53.055176 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.055101 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8\" (UID: \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" Jul 09 14:19:53.064128 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.064103 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4zwt\" (UniqueName: \"kubernetes.io/projected/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-kube-api-access-c4zwt\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8\" (UID: \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" Jul 09 14:19:53.131835 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.131801 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" Jul 09 14:19:53.268206 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.268181 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8"] Jul 09 14:19:53.270392 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:19:53.270357 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc27ee25b_9f01_4bc6_a509_67a61eaa1c1b.slice/crio-1d586dab8d51a49e954bd063d340ed9916d5de1c20ede9952047d144240151fb WatchSource:0}: Error finding container 1d586dab8d51a49e954bd063d340ed9916d5de1c20ede9952047d144240151fb: Status 404 returned error can't find the container with id 1d586dab8d51a49e954bd063d340ed9916d5de1c20ede9952047d144240151fb Jul 09 14:19:53.413742 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.413662 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4"] Jul 09 14:19:53.415846 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.415824 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" Jul 09 14:19:53.427074 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.427045 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4"] Jul 09 14:19:53.512529 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.512491 2581 generic.go:356] "Generic (PLEG): container finished" podID="c27ee25b-9f01-4bc6-a509-67a61eaa1c1b" containerID="88b31df32c73fcd4399e035d67bc18b17cea82a64094bb4bad773161ecae5736" exitCode=0 Jul 09 14:19:53.512693 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.512545 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" event={"ID":"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b","Type":"ContainerDied","Data":"88b31df32c73fcd4399e035d67bc18b17cea82a64094bb4bad773161ecae5736"} Jul 09 14:19:53.512693 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.512590 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" event={"ID":"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b","Type":"ContainerStarted","Data":"1d586dab8d51a49e954bd063d340ed9916d5de1c20ede9952047d144240151fb"} Jul 09 14:19:53.513125 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.513109 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 14:19:53.558261 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.558227 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqd67\" (UniqueName: \"kubernetes.io/projected/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-kube-api-access-bqd67\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4\" (UID: \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" Jul 09 14:19:53.558444 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.558270 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4\" (UID: \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" Jul 09 14:19:53.558444 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.558313 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4\" (UID: \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" Jul 09 14:19:53.659230 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.659191 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqd67\" (UniqueName: \"kubernetes.io/projected/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-kube-api-access-bqd67\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4\" (UID: \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" Jul 09 14:19:53.659230 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.659231 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4\" (UID: \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" Jul 09 14:19:53.659479 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.659262 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4\" (UID: \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" Jul 09 14:19:53.659663 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.659640 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4\" (UID: \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" Jul 09 14:19:53.659663 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.659654 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4\" (UID: \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" Jul 09 14:19:53.669237 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.669183 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqd67\" (UniqueName: \"kubernetes.io/projected/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-kube-api-access-bqd67\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4\" (UID: \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" Jul 09 14:19:53.741330 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.741295 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" Jul 09 14:19:53.874279 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:53.874247 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4"] Jul 09 14:19:53.877064 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:19:53.877036 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06ea3e13_372c_40f8_8c58_ce28e8fc64b6.slice/crio-e65562751600f8e8429b91332b0371d8d9fe3d7b926b4a093b74f0eb28b040a0 WatchSource:0}: Error finding container e65562751600f8e8429b91332b0371d8d9fe3d7b926b4a093b74f0eb28b040a0: Status 404 returned error can't find the container with id e65562751600f8e8429b91332b0371d8d9fe3d7b926b4a093b74f0eb28b040a0 Jul 09 14:19:54.214115 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.214040 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk"] Jul 09 14:19:54.216296 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.216280 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" Jul 09 14:19:54.226725 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.226699 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk"] Jul 09 14:19:54.365633 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.365589 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83e755e4-e8cd-4808-9c41-69afeee0f4f2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk\" (UID: \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" Jul 09 14:19:54.365828 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.365719 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6wsf\" (UniqueName: \"kubernetes.io/projected/83e755e4-e8cd-4808-9c41-69afeee0f4f2-kube-api-access-b6wsf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk\" (UID: \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" Jul 09 14:19:54.365828 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.365756 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83e755e4-e8cd-4808-9c41-69afeee0f4f2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk\" (UID: \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" Jul 09 14:19:54.466711 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.466622 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83e755e4-e8cd-4808-9c41-69afeee0f4f2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk\" (UID: \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" Jul 09 14:19:54.466711 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.466695 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6wsf\" (UniqueName: \"kubernetes.io/projected/83e755e4-e8cd-4808-9c41-69afeee0f4f2-kube-api-access-b6wsf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk\" (UID: \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" Jul 09 14:19:54.466895 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.466715 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83e755e4-e8cd-4808-9c41-69afeee0f4f2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk\" (UID: \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" Jul 09 14:19:54.467082 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.467061 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83e755e4-e8cd-4808-9c41-69afeee0f4f2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk\" (UID: \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" Jul 09 14:19:54.467127 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.467085 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83e755e4-e8cd-4808-9c41-69afeee0f4f2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk\" (UID: \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" Jul 09 14:19:54.478730 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.478705 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6wsf\" (UniqueName: \"kubernetes.io/projected/83e755e4-e8cd-4808-9c41-69afeee0f4f2-kube-api-access-b6wsf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk\" (UID: \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" Jul 09 14:19:54.519610 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.519574 2581 generic.go:356] "Generic (PLEG): container finished" podID="06ea3e13-372c-40f8-8c58-ce28e8fc64b6" containerID="c52c6dfa7b64ec55384390d30434e925281ebde3dd0df0c49834c40052b02b7f" exitCode=0 Jul 09 14:19:54.519767 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.519655 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" event={"ID":"06ea3e13-372c-40f8-8c58-ce28e8fc64b6","Type":"ContainerDied","Data":"c52c6dfa7b64ec55384390d30434e925281ebde3dd0df0c49834c40052b02b7f"} Jul 09 14:19:54.519767 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.519705 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" event={"ID":"06ea3e13-372c-40f8-8c58-ce28e8fc64b6","Type":"ContainerStarted","Data":"e65562751600f8e8429b91332b0371d8d9fe3d7b926b4a093b74f0eb28b040a0"} Jul 09 14:19:54.526727 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.526704 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" Jul 09 14:19:54.693643 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.693618 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk"] Jul 09 14:19:54.695894 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:19:54.695863 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83e755e4_e8cd_4808_9c41_69afeee0f4f2.slice/crio-dbb58e61573c9e7b49506b289c8d7944e45022a77d86e921766d39a111b4c09e WatchSource:0}: Error finding container dbb58e61573c9e7b49506b289c8d7944e45022a77d86e921766d39a111b4c09e: Status 404 returned error can't find the container with id dbb58e61573c9e7b49506b289c8d7944e45022a77d86e921766d39a111b4c09e Jul 09 14:19:54.814843 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.814813 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j"] Jul 09 14:19:54.818653 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.818628 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" Jul 09 14:19:54.826989 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.826935 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j"] Jul 09 14:19:54.971900 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.971865 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51ca537b-213b-4d1f-aba5-c20dcc94d90d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j\" (UID: \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" Jul 09 14:19:54.971900 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.971901 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51ca537b-213b-4d1f-aba5-c20dcc94d90d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j\" (UID: \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" Jul 09 14:19:54.972432 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:54.971921 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-997jt\" (UniqueName: \"kubernetes.io/projected/51ca537b-213b-4d1f-aba5-c20dcc94d90d-kube-api-access-997jt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j\" (UID: \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" Jul 09 14:19:55.073073 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.072988 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51ca537b-213b-4d1f-aba5-c20dcc94d90d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j\" (UID: \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" Jul 09 14:19:55.073073 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.073027 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51ca537b-213b-4d1f-aba5-c20dcc94d90d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j\" (UID: \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" Jul 09 14:19:55.073073 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.073047 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997jt\" (UniqueName: \"kubernetes.io/projected/51ca537b-213b-4d1f-aba5-c20dcc94d90d-kube-api-access-997jt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j\" (UID: \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" Jul 09 14:19:55.073388 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.073366 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51ca537b-213b-4d1f-aba5-c20dcc94d90d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j\" (UID: \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" Jul 09 14:19:55.073460 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.073419 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51ca537b-213b-4d1f-aba5-c20dcc94d90d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j\" (UID: \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" Jul 09 14:19:55.082380 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.082353 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-997jt\" (UniqueName: \"kubernetes.io/projected/51ca537b-213b-4d1f-aba5-c20dcc94d90d-kube-api-access-997jt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j\" (UID: \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" Jul 09 14:19:55.207473 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.207440 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" Jul 09 14:19:55.337969 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.337929 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j"] Jul 09 14:19:55.339813 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:19:55.339789 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51ca537b_213b_4d1f_aba5_c20dcc94d90d.slice/crio-bf9ce0502420a57932beddc2f25efd43bff057e2910489936b5e748d853efbd7 WatchSource:0}: Error finding container bf9ce0502420a57932beddc2f25efd43bff057e2910489936b5e748d853efbd7: Status 404 returned error can't find the container with id bf9ce0502420a57932beddc2f25efd43bff057e2910489936b5e748d853efbd7 Jul 09 14:19:55.527008 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.526971 2581 generic.go:356] "Generic (PLEG): container finished" podID="83e755e4-e8cd-4808-9c41-69afeee0f4f2" containerID="8b50fc4475b0c789eb11ca219b8a280a00d869843bd5aa6a0676ea92977766be" exitCode=0 Jul 09 14:19:55.527194 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.527010 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" event={"ID":"83e755e4-e8cd-4808-9c41-69afeee0f4f2","Type":"ContainerDied","Data":"8b50fc4475b0c789eb11ca219b8a280a00d869843bd5aa6a0676ea92977766be"} Jul 09 14:19:55.527194 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.527063 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" event={"ID":"83e755e4-e8cd-4808-9c41-69afeee0f4f2","Type":"ContainerStarted","Data":"dbb58e61573c9e7b49506b289c8d7944e45022a77d86e921766d39a111b4c09e"} Jul 09 14:19:55.529043 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.529012 2581 generic.go:356] "Generic (PLEG): container finished" podID="c27ee25b-9f01-4bc6-a509-67a61eaa1c1b" containerID="02b8ce04036268a1a3caf17b06de81c4a534ad9444d9c97128bf6d695ce5dd00" exitCode=0 Jul 09 14:19:55.529197 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.529103 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" event={"ID":"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b","Type":"ContainerDied","Data":"02b8ce04036268a1a3caf17b06de81c4a534ad9444d9c97128bf6d695ce5dd00"} Jul 09 14:19:55.530836 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.530811 2581 generic.go:356] "Generic (PLEG): container finished" podID="51ca537b-213b-4d1f-aba5-c20dcc94d90d" containerID="ab36605490bc89d0908d3c39f9b2b291e5e77e850d757bfdc8e91db6954e9761" exitCode=0 Jul 09 14:19:55.530924 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.530871 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" event={"ID":"51ca537b-213b-4d1f-aba5-c20dcc94d90d","Type":"ContainerDied","Data":"ab36605490bc89d0908d3c39f9b2b291e5e77e850d757bfdc8e91db6954e9761"} Jul 09 14:19:55.530992 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:55.530926 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" event={"ID":"51ca537b-213b-4d1f-aba5-c20dcc94d90d","Type":"ContainerStarted","Data":"bf9ce0502420a57932beddc2f25efd43bff057e2910489936b5e748d853efbd7"} Jul 09 14:19:56.538782 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:56.538752 2581 generic.go:356] "Generic (PLEG): container finished" podID="c27ee25b-9f01-4bc6-a509-67a61eaa1c1b" containerID="72a66c8b23e587c01026cd303ece0788dfd301f69dbe4d35090967d69f33d092" exitCode=0 Jul 09 14:19:56.539266 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:56.538844 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" event={"ID":"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b","Type":"ContainerDied","Data":"72a66c8b23e587c01026cd303ece0788dfd301f69dbe4d35090967d69f33d092"} Jul 09 14:19:56.540560 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:56.540536 2581 generic.go:356] "Generic (PLEG): container finished" podID="51ca537b-213b-4d1f-aba5-c20dcc94d90d" containerID="d0ac7cb38ce5f63dbb0dec2135c5ee45641864587bafaf5e99919b08388bb2f0" exitCode=0 Jul 09 14:19:56.540683 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:56.540617 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" event={"ID":"51ca537b-213b-4d1f-aba5-c20dcc94d90d","Type":"ContainerDied","Data":"d0ac7cb38ce5f63dbb0dec2135c5ee45641864587bafaf5e99919b08388bb2f0"} Jul 09 14:19:57.547216 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:57.547179 2581 generic.go:356] "Generic (PLEG): container finished" podID="06ea3e13-372c-40f8-8c58-ce28e8fc64b6" containerID="be87453d764e4a41453caf3bd89775f572d525a6b21d3f6097877d7b9425b5df" exitCode=0 Jul 09 14:19:57.547618 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:57.547261 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" event={"ID":"06ea3e13-372c-40f8-8c58-ce28e8fc64b6","Type":"ContainerDied","Data":"be87453d764e4a41453caf3bd89775f572d525a6b21d3f6097877d7b9425b5df"} Jul 09 14:19:57.549404 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:57.549383 2581 generic.go:356] "Generic (PLEG): container finished" podID="51ca537b-213b-4d1f-aba5-c20dcc94d90d" containerID="0935f701059c498dee50914d04330a4eb90c736e049c3af5734eeabb3344427a" exitCode=0 Jul 09 14:19:57.549508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:57.549457 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" event={"ID":"51ca537b-213b-4d1f-aba5-c20dcc94d90d","Type":"ContainerDied","Data":"0935f701059c498dee50914d04330a4eb90c736e049c3af5734eeabb3344427a"} Jul 09 14:19:57.815235 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:57.815209 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" Jul 09 14:19:57.899703 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:57.899673 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-bundle\") pod \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\" (UID: \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\") " Jul 09 14:19:57.899855 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:57.899730 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-util\") pod \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\" (UID: \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\") " Jul 09 14:19:57.899855 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:57.899774 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4zwt\" (UniqueName: \"kubernetes.io/projected/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-kube-api-access-c4zwt\") pod \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\" (UID: \"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b\") " Jul 09 14:19:57.900170 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:57.900143 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-bundle" (OuterVolumeSpecName: "bundle") pod "c27ee25b-9f01-4bc6-a509-67a61eaa1c1b" (UID: "c27ee25b-9f01-4bc6-a509-67a61eaa1c1b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:19:57.901936 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:57.901911 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-kube-api-access-c4zwt" (OuterVolumeSpecName: "kube-api-access-c4zwt") pod "c27ee25b-9f01-4bc6-a509-67a61eaa1c1b" (UID: "c27ee25b-9f01-4bc6-a509-67a61eaa1c1b"). InnerVolumeSpecName "kube-api-access-c4zwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:19:57.905303 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:57.905268 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-util" (OuterVolumeSpecName: "util") pod "c27ee25b-9f01-4bc6-a509-67a61eaa1c1b" (UID: "c27ee25b-9f01-4bc6-a509-67a61eaa1c1b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:19:58.000554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.000522 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-util\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:19:58.000554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.000551 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4zwt\" (UniqueName: \"kubernetes.io/projected/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-kube-api-access-c4zwt\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:19:58.000554 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.000561 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c27ee25b-9f01-4bc6-a509-67a61eaa1c1b-bundle\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:19:58.556507 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.556468 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" event={"ID":"c27ee25b-9f01-4bc6-a509-67a61eaa1c1b","Type":"ContainerDied","Data":"1d586dab8d51a49e954bd063d340ed9916d5de1c20ede9952047d144240151fb"} Jul 09 14:19:58.557015 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.556532 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d586dab8d51a49e954bd063d340ed9916d5de1c20ede9952047d144240151fb" Jul 09 14:19:58.557015 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.556480 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8" Jul 09 14:19:58.558735 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.558708 2581 generic.go:356] "Generic (PLEG): container finished" podID="06ea3e13-372c-40f8-8c58-ce28e8fc64b6" containerID="fb4892ae1b0e30d7dfea64be30f128dd4edc46456ac7f536bf88d56fbdac36d2" exitCode=0 Jul 09 14:19:58.558872 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.558784 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" event={"ID":"06ea3e13-372c-40f8-8c58-ce28e8fc64b6","Type":"ContainerDied","Data":"fb4892ae1b0e30d7dfea64be30f128dd4edc46456ac7f536bf88d56fbdac36d2"} Jul 09 14:19:58.560694 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.560669 2581 generic.go:356] "Generic (PLEG): container finished" podID="83e755e4-e8cd-4808-9c41-69afeee0f4f2" containerID="2a069c43056ed5bb16196611690af6c8a7414cadcccebcfda79c3428be494092" exitCode=0 Jul 09 14:19:58.560822 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.560722 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" event={"ID":"83e755e4-e8cd-4808-9c41-69afeee0f4f2","Type":"ContainerDied","Data":"2a069c43056ed5bb16196611690af6c8a7414cadcccebcfda79c3428be494092"} Jul 09 14:19:58.695074 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.695043 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" Jul 09 14:19:58.807475 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.807383 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51ca537b-213b-4d1f-aba5-c20dcc94d90d-bundle\") pod \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\" (UID: \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\") " Jul 09 14:19:58.807475 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.807458 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-997jt\" (UniqueName: \"kubernetes.io/projected/51ca537b-213b-4d1f-aba5-c20dcc94d90d-kube-api-access-997jt\") pod \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\" (UID: \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\") " Jul 09 14:19:58.807722 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.807514 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51ca537b-213b-4d1f-aba5-c20dcc94d90d-util\") pod \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\" (UID: \"51ca537b-213b-4d1f-aba5-c20dcc94d90d\") " Jul 09 14:19:58.807906 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.807877 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ca537b-213b-4d1f-aba5-c20dcc94d90d-bundle" (OuterVolumeSpecName: "bundle") pod "51ca537b-213b-4d1f-aba5-c20dcc94d90d" (UID: "51ca537b-213b-4d1f-aba5-c20dcc94d90d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:19:58.809529 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.809503 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51ca537b-213b-4d1f-aba5-c20dcc94d90d-kube-api-access-997jt" (OuterVolumeSpecName: "kube-api-access-997jt") pod "51ca537b-213b-4d1f-aba5-c20dcc94d90d" (UID: "51ca537b-213b-4d1f-aba5-c20dcc94d90d"). InnerVolumeSpecName "kube-api-access-997jt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:19:58.812874 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.812840 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ca537b-213b-4d1f-aba5-c20dcc94d90d-util" (OuterVolumeSpecName: "util") pod "51ca537b-213b-4d1f-aba5-c20dcc94d90d" (UID: "51ca537b-213b-4d1f-aba5-c20dcc94d90d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:19:58.908593 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.908550 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-997jt\" (UniqueName: \"kubernetes.io/projected/51ca537b-213b-4d1f-aba5-c20dcc94d90d-kube-api-access-997jt\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:19:58.908593 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.908587 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51ca537b-213b-4d1f-aba5-c20dcc94d90d-util\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:19:58.908593 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:58.908597 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51ca537b-213b-4d1f-aba5-c20dcc94d90d-bundle\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:19:59.567531 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.567495 2581 generic.go:356] "Generic (PLEG): container finished" podID="83e755e4-e8cd-4808-9c41-69afeee0f4f2" containerID="e42f24152f6cf0fe80702933a739a02e0fa1c24f179c334c5580df61f774086e" exitCode=0 Jul 09 14:19:59.567970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.567578 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" event={"ID":"83e755e4-e8cd-4808-9c41-69afeee0f4f2","Type":"ContainerDied","Data":"e42f24152f6cf0fe80702933a739a02e0fa1c24f179c334c5580df61f774086e"} Jul 09 14:19:59.569446 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.569413 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" event={"ID":"51ca537b-213b-4d1f-aba5-c20dcc94d90d","Type":"ContainerDied","Data":"bf9ce0502420a57932beddc2f25efd43bff057e2910489936b5e748d853efbd7"} Jul 09 14:19:59.569539 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.569474 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf9ce0502420a57932beddc2f25efd43bff057e2910489936b5e748d853efbd7" Jul 09 14:19:59.569539 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.569428 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j" Jul 09 14:19:59.699387 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.699353 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" Jul 09 14:19:59.818181 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.818091 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqd67\" (UniqueName: \"kubernetes.io/projected/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-kube-api-access-bqd67\") pod \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\" (UID: \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\") " Jul 09 14:19:59.818181 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.818163 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-bundle\") pod \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\" (UID: \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\") " Jul 09 14:19:59.818436 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.818212 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-util\") pod \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\" (UID: \"06ea3e13-372c-40f8-8c58-ce28e8fc64b6\") " Jul 09 14:19:59.818769 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.818733 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-bundle" (OuterVolumeSpecName: "bundle") pod "06ea3e13-372c-40f8-8c58-ce28e8fc64b6" (UID: "06ea3e13-372c-40f8-8c58-ce28e8fc64b6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:19:59.820273 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.820245 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-kube-api-access-bqd67" (OuterVolumeSpecName: "kube-api-access-bqd67") pod "06ea3e13-372c-40f8-8c58-ce28e8fc64b6" (UID: "06ea3e13-372c-40f8-8c58-ce28e8fc64b6"). InnerVolumeSpecName "kube-api-access-bqd67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:19:59.822839 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.822817 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-util" (OuterVolumeSpecName: "util") pod "06ea3e13-372c-40f8-8c58-ce28e8fc64b6" (UID: "06ea3e13-372c-40f8-8c58-ce28e8fc64b6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:19:59.919198 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.919165 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-bundle\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:19:59.919198 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.919192 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-util\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:19:59.919198 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:19:59.919202 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqd67\" (UniqueName: \"kubernetes.io/projected/06ea3e13-372c-40f8-8c58-ce28e8fc64b6-kube-api-access-bqd67\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:20:00.576312 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:00.576277 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" event={"ID":"06ea3e13-372c-40f8-8c58-ce28e8fc64b6","Type":"ContainerDied","Data":"e65562751600f8e8429b91332b0371d8d9fe3d7b926b4a093b74f0eb28b040a0"} Jul 09 14:20:00.576312 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:00.576317 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4" Jul 09 14:20:00.576765 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:00.576336 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e65562751600f8e8429b91332b0371d8d9fe3d7b926b4a093b74f0eb28b040a0" Jul 09 14:20:00.707988 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:00.707946 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" Jul 09 14:20:00.827078 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:00.826981 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83e755e4-e8cd-4808-9c41-69afeee0f4f2-util\") pod \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\" (UID: \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\") " Jul 09 14:20:00.827078 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:00.827029 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83e755e4-e8cd-4808-9c41-69afeee0f4f2-bundle\") pod \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\" (UID: \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\") " Jul 09 14:20:00.827322 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:00.827098 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6wsf\" (UniqueName: \"kubernetes.io/projected/83e755e4-e8cd-4808-9c41-69afeee0f4f2-kube-api-access-b6wsf\") pod \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\" (UID: \"83e755e4-e8cd-4808-9c41-69afeee0f4f2\") " Jul 09 14:20:00.827521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:00.827497 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e755e4-e8cd-4808-9c41-69afeee0f4f2-bundle" (OuterVolumeSpecName: "bundle") pod "83e755e4-e8cd-4808-9c41-69afeee0f4f2" (UID: "83e755e4-e8cd-4808-9c41-69afeee0f4f2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:20:00.829213 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:00.829188 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83e755e4-e8cd-4808-9c41-69afeee0f4f2-kube-api-access-b6wsf" (OuterVolumeSpecName: "kube-api-access-b6wsf") pod "83e755e4-e8cd-4808-9c41-69afeee0f4f2" (UID: "83e755e4-e8cd-4808-9c41-69afeee0f4f2"). InnerVolumeSpecName "kube-api-access-b6wsf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:20:00.831810 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:00.831771 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83e755e4-e8cd-4808-9c41-69afeee0f4f2-util" (OuterVolumeSpecName: "util") pod "83e755e4-e8cd-4808-9c41-69afeee0f4f2" (UID: "83e755e4-e8cd-4808-9c41-69afeee0f4f2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:20:00.927847 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:00.927808 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83e755e4-e8cd-4808-9c41-69afeee0f4f2-util\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:20:00.927847 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:00.927838 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83e755e4-e8cd-4808-9c41-69afeee0f4f2-bundle\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:20:00.927847 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:00.927852 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6wsf\" (UniqueName: \"kubernetes.io/projected/83e755e4-e8cd-4808-9c41-69afeee0f4f2-kube-api-access-b6wsf\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:20:01.582813 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:01.582774 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" event={"ID":"83e755e4-e8cd-4808-9c41-69afeee0f4f2","Type":"ContainerDied","Data":"dbb58e61573c9e7b49506b289c8d7944e45022a77d86e921766d39a111b4c09e"} Jul 09 14:20:01.583186 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:01.582827 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbb58e61573c9e7b49506b289c8d7944e45022a77d86e921766d39a111b4c09e" Jul 09 14:20:01.583186 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:01.582794 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk" Jul 09 14:20:19.086212 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086173 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-p7xtb"] Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086632 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ca537b-213b-4d1f-aba5-c20dcc94d90d" containerName="util" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086650 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ca537b-213b-4d1f-aba5-c20dcc94d90d" containerName="util" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086666 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ca537b-213b-4d1f-aba5-c20dcc94d90d" containerName="extract" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086675 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ca537b-213b-4d1f-aba5-c20dcc94d90d" containerName="extract" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086690 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83e755e4-e8cd-4808-9c41-69afeee0f4f2" containerName="util" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086699 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="83e755e4-e8cd-4808-9c41-69afeee0f4f2" containerName="util" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086710 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83e755e4-e8cd-4808-9c41-69afeee0f4f2" containerName="pull" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086717 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="83e755e4-e8cd-4808-9c41-69afeee0f4f2" containerName="pull" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086730 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06ea3e13-372c-40f8-8c58-ce28e8fc64b6" containerName="util" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086738 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="06ea3e13-372c-40f8-8c58-ce28e8fc64b6" containerName="util" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086754 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c27ee25b-9f01-4bc6-a509-67a61eaa1c1b" containerName="pull" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086761 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c27ee25b-9f01-4bc6-a509-67a61eaa1c1b" containerName="pull" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086770 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ca537b-213b-4d1f-aba5-c20dcc94d90d" containerName="pull" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086777 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ca537b-213b-4d1f-aba5-c20dcc94d90d" containerName="pull" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086790 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c27ee25b-9f01-4bc6-a509-67a61eaa1c1b" containerName="extract" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086797 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c27ee25b-9f01-4bc6-a509-67a61eaa1c1b" containerName="extract" Jul 09 14:20:19.086798 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086805 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83e755e4-e8cd-4808-9c41-69afeee0f4f2" containerName="extract" Jul 09 14:20:19.087612 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086813 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="83e755e4-e8cd-4808-9c41-69afeee0f4f2" containerName="extract" Jul 09 14:20:19.087612 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086831 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c27ee25b-9f01-4bc6-a509-67a61eaa1c1b" containerName="util" Jul 09 14:20:19.087612 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086838 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c27ee25b-9f01-4bc6-a509-67a61eaa1c1b" containerName="util" Jul 09 14:20:19.087612 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086848 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06ea3e13-372c-40f8-8c58-ce28e8fc64b6" containerName="extract" Jul 09 14:20:19.087612 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086856 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="06ea3e13-372c-40f8-8c58-ce28e8fc64b6" containerName="extract" Jul 09 14:20:19.087612 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086867 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06ea3e13-372c-40f8-8c58-ce28e8fc64b6" containerName="pull" Jul 09 14:20:19.087612 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.086875 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="06ea3e13-372c-40f8-8c58-ce28e8fc64b6" containerName="pull" Jul 09 14:20:19.087612 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.087000 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c27ee25b-9f01-4bc6-a509-67a61eaa1c1b" containerName="extract" Jul 09 14:20:19.087612 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.087015 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="51ca537b-213b-4d1f-aba5-c20dcc94d90d" containerName="extract" Jul 09 14:20:19.087612 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.087027 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="83e755e4-e8cd-4808-9c41-69afeee0f4f2" containerName="extract" Jul 09 14:20:19.087612 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.087039 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="06ea3e13-372c-40f8-8c58-ce28e8fc64b6" containerName="extract" Jul 09 14:20:19.091927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.091907 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p7xtb" Jul 09 14:20:19.095847 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.095826 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 14:20:19.098195 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.098175 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 14:20:19.098320 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.098235 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 14:20:19.098320 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.098175 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-pl8px\"" Jul 09 14:20:19.106520 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.106494 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-p7xtb"] Jul 09 14:20:19.185810 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.185762 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt4n6\" (UniqueName: \"kubernetes.io/projected/5f30310c-5fc0-4d28-b58f-0903c046070b-kube-api-access-pt4n6\") pod \"dns-operator-controller-manager-864dd78c4c-p7xtb\" (UID: \"5f30310c-5fc0-4d28-b58f-0903c046070b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p7xtb" Jul 09 14:20:19.287038 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.286977 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt4n6\" (UniqueName: \"kubernetes.io/projected/5f30310c-5fc0-4d28-b58f-0903c046070b-kube-api-access-pt4n6\") pod \"dns-operator-controller-manager-864dd78c4c-p7xtb\" (UID: \"5f30310c-5fc0-4d28-b58f-0903c046070b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p7xtb" Jul 09 14:20:19.301199 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.301172 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt4n6\" (UniqueName: \"kubernetes.io/projected/5f30310c-5fc0-4d28-b58f-0903c046070b-kube-api-access-pt4n6\") pod \"dns-operator-controller-manager-864dd78c4c-p7xtb\" (UID: \"5f30310c-5fc0-4d28-b58f-0903c046070b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p7xtb" Jul 09 14:20:19.401851 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.401808 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p7xtb" Jul 09 14:20:19.545991 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.545962 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-p7xtb"] Jul 09 14:20:19.547718 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:20:19.547691 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f30310c_5fc0_4d28_b58f_0903c046070b.slice/crio-0320338107e492fec8cb22a28ea102d87ad9fdbb689305ae8c08cd9893bed7f9 WatchSource:0}: Error finding container 0320338107e492fec8cb22a28ea102d87ad9fdbb689305ae8c08cd9893bed7f9: Status 404 returned error can't find the container with id 0320338107e492fec8cb22a28ea102d87ad9fdbb689305ae8c08cd9893bed7f9 Jul 09 14:20:19.672336 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:19.672249 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p7xtb" event={"ID":"5f30310c-5fc0-4d28-b58f-0903c046070b","Type":"ContainerStarted","Data":"0320338107e492fec8cb22a28ea102d87ad9fdbb689305ae8c08cd9893bed7f9"} Jul 09 14:20:22.689510 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:22.689470 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p7xtb" event={"ID":"5f30310c-5fc0-4d28-b58f-0903c046070b","Type":"ContainerStarted","Data":"895fbfad4063f0dacafbe94d66395415a3cc89bbc80320b585731291dc6fc0fa"} Jul 09 14:20:22.689941 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:22.689665 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p7xtb" Jul 09 14:20:22.716474 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:22.716426 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p7xtb" podStartSLOduration=1.199889504 podStartE2EDuration="3.716410135s" totalImagesPullingTime="2.516520631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:20:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:20:22.714925107 +0000 UTC m=+342.838064492" watchObservedRunningTime="2026-07-09 14:20:22.716410135 +0000 UTC m=+342.839549514" Jul 09 14:20:33.697704 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:33.697616 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p7xtb" Jul 09 14:20:43.073618 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.073584 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f569c684-cmlq7"] Jul 09 14:20:43.077015 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.076992 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.088836 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.088809 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f569c684-cmlq7"] Jul 09 14:20:43.210917 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.210875 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-trusted-ca-bundle\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.210917 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.210919 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-service-ca\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.211142 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.211039 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6znb9\" (UniqueName: \"kubernetes.io/projected/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-kube-api-access-6znb9\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.211142 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.211086 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-oauth-serving-cert\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.211142 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.211109 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-console-serving-cert\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.211239 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.211153 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-console-oauth-config\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.211239 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.211180 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-console-config\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.312618 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.312579 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-trusted-ca-bundle\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.312618 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.312618 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-service-ca\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.312879 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.312668 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6znb9\" (UniqueName: \"kubernetes.io/projected/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-kube-api-access-6znb9\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.312879 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.312694 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-oauth-serving-cert\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.312879 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.312711 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-console-serving-cert\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.312879 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.312737 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-console-oauth-config\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.312879 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.312751 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-console-config\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.313525 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.313465 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-oauth-serving-cert\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.313641 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.313547 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-console-config\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.313641 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.313625 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-service-ca\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.313872 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.313850 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-trusted-ca-bundle\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.315253 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.315223 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-console-oauth-config\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.315415 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.315396 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-console-serving-cert\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.321861 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.321832 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6znb9\" (UniqueName: \"kubernetes.io/projected/30fd575c-e61c-4ff2-9608-4385bd0bcb4b-kube-api-access-6znb9\") pod \"console-7f569c684-cmlq7\" (UID: \"30fd575c-e61c-4ff2-9608-4385bd0bcb4b\") " pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.387304 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.387274 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:43.531326 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.531292 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f569c684-cmlq7"] Jul 09 14:20:43.534331 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:20:43.534297 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30fd575c_e61c_4ff2_9608_4385bd0bcb4b.slice/crio-cea5b2744313a621c37a2163cbe9cdf18ec178da5261444d361205210c01bc9b WatchSource:0}: Error finding container cea5b2744313a621c37a2163cbe9cdf18ec178da5261444d361205210c01bc9b: Status 404 returned error can't find the container with id cea5b2744313a621c37a2163cbe9cdf18ec178da5261444d361205210c01bc9b Jul 09 14:20:43.803204 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.803113 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f569c684-cmlq7" event={"ID":"30fd575c-e61c-4ff2-9608-4385bd0bcb4b","Type":"ContainerStarted","Data":"2a6cda26e7f4b120f57b2ca3c18e5b5d955a8ca55104c8ef32c57cdcac9e5e17"} Jul 09 14:20:43.803204 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.803169 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f569c684-cmlq7" event={"ID":"30fd575c-e61c-4ff2-9608-4385bd0bcb4b","Type":"ContainerStarted","Data":"cea5b2744313a621c37a2163cbe9cdf18ec178da5261444d361205210c01bc9b"} Jul 09 14:20:43.827061 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:43.827010 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f569c684-cmlq7" podStartSLOduration=0.826994416 podStartE2EDuration="826.994416ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:20:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:20:43.826132477 +0000 UTC m=+363.949271878" watchObservedRunningTime="2026-07-09 14:20:43.826994416 +0000 UTC m=+363.950133856" Jul 09 14:20:53.388106 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:53.388064 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:53.388674 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:53.388128 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:53.392937 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:53.392916 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:53.855448 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:53.855421 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f569c684-cmlq7" Jul 09 14:20:53.911489 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:20:53.911459 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6ff87f966f-qmfd5"] Jul 09 14:21:04.615289 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.615256 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn"] Jul 09 14:21:04.618431 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.618411 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.621453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.621437 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-flx9w\"" Jul 09 14:21:04.631473 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.631429 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn"] Jul 09 14:21:04.696993 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.696929 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.697151 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.697061 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.697151 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.697099 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.697151 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.697117 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.697256 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.697167 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.697256 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.697199 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.697256 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.697219 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.697256 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.697249 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.697374 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.697270 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jl45\" (UniqueName: \"kubernetes.io/projected/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-kube-api-access-9jl45\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.798635 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.798591 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.798635 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.798638 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.798886 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.798661 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jl45\" (UniqueName: \"kubernetes.io/projected/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-kube-api-access-9jl45\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.798886 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.798688 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.798886 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.798775 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.798886 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.798807 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.798886 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.798827 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.799168 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.798900 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.799168 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.798981 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.799168 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.799058 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.799168 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.799110 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.799363 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.799236 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.799415 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.799393 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.799619 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.799599 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.801143 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.801122 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.801416 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.801399 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.807139 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.807118 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.807356 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.807338 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jl45\" (UniqueName: \"kubernetes.io/projected/34d9a37b-862d-4a2a-90f8-72c2e07cf9f9-kube-api-access-9jl45\") pod \"maas-default-gateway-openshift-default-56654dcf7-6sngn\" (UID: \"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:04.929225 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:04.929138 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:05.072825 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:05.072793 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn"] Jul 09 14:21:05.074419 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:21:05.074391 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d9a37b_862d_4a2a_90f8_72c2e07cf9f9.slice/crio-470627d3ce8041b3501a34144cadecf4a3291a23040b76919da7f7803930d555 WatchSource:0}: Error finding container 470627d3ce8041b3501a34144cadecf4a3291a23040b76919da7f7803930d555: Status 404 returned error can't find the container with id 470627d3ce8041b3501a34144cadecf4a3291a23040b76919da7f7803930d555 Jul 09 14:21:05.076479 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:05.076447 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:21:05.076541 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:05.076513 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:21:05.076583 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:05.076542 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:21:05.914460 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:05.914424 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" event={"ID":"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9","Type":"ContainerStarted","Data":"57036640473fab22f8225bcfa0552496b3f55727dd4c1d861eb47fe4ffdc26ad"} Jul 09 14:21:05.914828 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:05.914478 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" event={"ID":"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9","Type":"ContainerStarted","Data":"470627d3ce8041b3501a34144cadecf4a3291a23040b76919da7f7803930d555"} Jul 09 14:21:05.929814 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:05.929792 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:05.948470 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:05.948414 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" podStartSLOduration=1.9484009100000002 podStartE2EDuration="1.94840091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:21:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:05.945839904 +0000 UTC m=+386.068979305" watchObservedRunningTime="2026-07-09 14:21:05.94840091 +0000 UTC m=+386.071540272" Jul 09 14:21:06.934888 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:06.934850 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:07.925402 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:07.925365 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:07.926349 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:07.926331 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:21:18.932927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:18.932878 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6ff87f966f-qmfd5" podUID="036b5417-59df-45e9-8f35-3f1d729b4281" containerName="console" containerID="cri-o://caa38c67caff92d65ba0dd559205054eb3fbd8f0aedba0c9f7c9f1ab5da2ead8" gracePeriod=15 Jul 09 14:21:19.189457 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.189397 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:21:19.336416 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.336393 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-oauth-serving-cert\") pod \"036b5417-59df-45e9-8f35-3f1d729b4281\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " Jul 09 14:21:19.336489 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.336446 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-trusted-ca-bundle\") pod \"036b5417-59df-45e9-8f35-3f1d729b4281\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " Jul 09 14:21:19.336489 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.336483 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-console-config\") pod \"036b5417-59df-45e9-8f35-3f1d729b4281\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " Jul 09 14:21:19.336558 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.336507 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/036b5417-59df-45e9-8f35-3f1d729b4281-console-oauth-config\") pod \"036b5417-59df-45e9-8f35-3f1d729b4281\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " Jul 09 14:21:19.336558 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.336539 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-service-ca\") pod \"036b5417-59df-45e9-8f35-3f1d729b4281\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " Jul 09 14:21:19.336632 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.336561 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/036b5417-59df-45e9-8f35-3f1d729b4281-console-serving-cert\") pod \"036b5417-59df-45e9-8f35-3f1d729b4281\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " Jul 09 14:21:19.336632 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.336608 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fv8h9\" (UniqueName: \"kubernetes.io/projected/036b5417-59df-45e9-8f35-3f1d729b4281-kube-api-access-fv8h9\") pod \"036b5417-59df-45e9-8f35-3f1d729b4281\" (UID: \"036b5417-59df-45e9-8f35-3f1d729b4281\") " Jul 09 14:21:19.336806 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.336777 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "036b5417-59df-45e9-8f35-3f1d729b4281" (UID: "036b5417-59df-45e9-8f35-3f1d729b4281"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:21:19.336927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.336903 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-oauth-serving-cert\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:21:19.336927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.336900 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-console-config" (OuterVolumeSpecName: "console-config") pod "036b5417-59df-45e9-8f35-3f1d729b4281" (UID: "036b5417-59df-45e9-8f35-3f1d729b4281"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:21:19.337064 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.336984 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-service-ca" (OuterVolumeSpecName: "service-ca") pod "036b5417-59df-45e9-8f35-3f1d729b4281" (UID: "036b5417-59df-45e9-8f35-3f1d729b4281"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:21:19.337064 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.336998 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "036b5417-59df-45e9-8f35-3f1d729b4281" (UID: "036b5417-59df-45e9-8f35-3f1d729b4281"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:21:19.338825 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.338796 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/036b5417-59df-45e9-8f35-3f1d729b4281-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "036b5417-59df-45e9-8f35-3f1d729b4281" (UID: "036b5417-59df-45e9-8f35-3f1d729b4281"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:21:19.339254 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.339234 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/036b5417-59df-45e9-8f35-3f1d729b4281-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "036b5417-59df-45e9-8f35-3f1d729b4281" (UID: "036b5417-59df-45e9-8f35-3f1d729b4281"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:21:19.339320 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.339258 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/036b5417-59df-45e9-8f35-3f1d729b4281-kube-api-access-fv8h9" (OuterVolumeSpecName: "kube-api-access-fv8h9") pod "036b5417-59df-45e9-8f35-3f1d729b4281" (UID: "036b5417-59df-45e9-8f35-3f1d729b4281"). InnerVolumeSpecName "kube-api-access-fv8h9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:21:19.371453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.371424 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:21:19.371765 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.371753 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="036b5417-59df-45e9-8f35-3f1d729b4281" containerName="console" Jul 09 14:21:19.371808 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.371766 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="036b5417-59df-45e9-8f35-3f1d729b4281" containerName="console" Jul 09 14:21:19.371914 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.371902 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="036b5417-59df-45e9-8f35-3f1d729b4281" containerName="console" Jul 09 14:21:19.375263 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.375246 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-zsj7t" Jul 09 14:21:19.378017 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.377996 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 14:21:19.378109 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.378015 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gcv6t\"" Jul 09 14:21:19.385575 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.385551 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:21:19.420026 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.419990 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:21:19.437976 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.437931 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ncq5\" (UniqueName: \"kubernetes.io/projected/3defbe4a-fb5a-4671-a840-0ee2b6c8e696-kube-api-access-6ncq5\") pod \"limitador-limitador-54d6d988d5-zsj7t\" (UID: \"3defbe4a-fb5a-4671-a840-0ee2b6c8e696\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zsj7t" Jul 09 14:21:19.438113 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.438009 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3defbe4a-fb5a-4671-a840-0ee2b6c8e696-config-file\") pod \"limitador-limitador-54d6d988d5-zsj7t\" (UID: \"3defbe4a-fb5a-4671-a840-0ee2b6c8e696\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zsj7t" Jul 09 14:21:19.438113 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.438073 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-trusted-ca-bundle\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:21:19.438113 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.438086 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-console-config\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:21:19.438113 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.438100 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/036b5417-59df-45e9-8f35-3f1d729b4281-console-oauth-config\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:21:19.438113 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.438108 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/036b5417-59df-45e9-8f35-3f1d729b4281-service-ca\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:21:19.438266 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.438116 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/036b5417-59df-45e9-8f35-3f1d729b4281-console-serving-cert\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:21:19.438266 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.438125 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fv8h9\" (UniqueName: \"kubernetes.io/projected/036b5417-59df-45e9-8f35-3f1d729b4281-kube-api-access-fv8h9\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:21:19.538794 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.538695 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ncq5\" (UniqueName: \"kubernetes.io/projected/3defbe4a-fb5a-4671-a840-0ee2b6c8e696-kube-api-access-6ncq5\") pod \"limitador-limitador-54d6d988d5-zsj7t\" (UID: \"3defbe4a-fb5a-4671-a840-0ee2b6c8e696\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zsj7t" Jul 09 14:21:19.538994 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.538859 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3defbe4a-fb5a-4671-a840-0ee2b6c8e696-config-file\") pod \"limitador-limitador-54d6d988d5-zsj7t\" (UID: \"3defbe4a-fb5a-4671-a840-0ee2b6c8e696\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zsj7t" Jul 09 14:21:19.539533 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.539506 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3defbe4a-fb5a-4671-a840-0ee2b6c8e696-config-file\") pod \"limitador-limitador-54d6d988d5-zsj7t\" (UID: \"3defbe4a-fb5a-4671-a840-0ee2b6c8e696\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zsj7t" Jul 09 14:21:19.547695 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.547671 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ncq5\" (UniqueName: \"kubernetes.io/projected/3defbe4a-fb5a-4671-a840-0ee2b6c8e696-kube-api-access-6ncq5\") pod \"limitador-limitador-54d6d988d5-zsj7t\" (UID: \"3defbe4a-fb5a-4671-a840-0ee2b6c8e696\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zsj7t" Jul 09 14:21:19.685906 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.685860 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-zsj7t" Jul 09 14:21:19.820841 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.820812 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:21:19.822649 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:21:19.822617 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3defbe4a_fb5a_4671_a840_0ee2b6c8e696.slice/crio-eefd1487dd0b801b5beaeca4f6cd0c41488b447ff486ddf5196517fea8fda482 WatchSource:0}: Error finding container eefd1487dd0b801b5beaeca4f6cd0c41488b447ff486ddf5196517fea8fda482: Status 404 returned error can't find the container with id eefd1487dd0b801b5beaeca4f6cd0c41488b447ff486ddf5196517fea8fda482 Jul 09 14:21:19.986274 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.986237 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-zsj7t" event={"ID":"3defbe4a-fb5a-4671-a840-0ee2b6c8e696","Type":"ContainerStarted","Data":"eefd1487dd0b801b5beaeca4f6cd0c41488b447ff486ddf5196517fea8fda482"} Jul 09 14:21:19.987550 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.987527 2581 generic.go:356] "Generic (PLEG): container finished" podID="036b5417-59df-45e9-8f35-3f1d729b4281" containerID="caa38c67caff92d65ba0dd559205054eb3fbd8f0aedba0c9f7c9f1ab5da2ead8" exitCode=0 Jul 09 14:21:19.987665 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.987561 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff87f966f-qmfd5" event={"ID":"036b5417-59df-45e9-8f35-3f1d729b4281","Type":"ContainerDied","Data":"caa38c67caff92d65ba0dd559205054eb3fbd8f0aedba0c9f7c9f1ab5da2ead8"} Jul 09 14:21:19.987665 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.987576 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff87f966f-qmfd5" Jul 09 14:21:19.987665 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.987589 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff87f966f-qmfd5" event={"ID":"036b5417-59df-45e9-8f35-3f1d729b4281","Type":"ContainerDied","Data":"a6e26afcffd0f58a1ccff6a81011cf140e2db3111c748e7ba34f2995b615e0d6"} Jul 09 14:21:19.987665 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.987612 2581 scope.go:117] "RemoveContainer" containerID="caa38c67caff92d65ba0dd559205054eb3fbd8f0aedba0c9f7c9f1ab5da2ead8" Jul 09 14:21:19.996922 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.996905 2581 scope.go:117] "RemoveContainer" containerID="caa38c67caff92d65ba0dd559205054eb3fbd8f0aedba0c9f7c9f1ab5da2ead8" Jul 09 14:21:19.997223 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:21:19.997199 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caa38c67caff92d65ba0dd559205054eb3fbd8f0aedba0c9f7c9f1ab5da2ead8\": container with ID starting with caa38c67caff92d65ba0dd559205054eb3fbd8f0aedba0c9f7c9f1ab5da2ead8 not found: ID does not exist" containerID="caa38c67caff92d65ba0dd559205054eb3fbd8f0aedba0c9f7c9f1ab5da2ead8" Jul 09 14:21:19.997316 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:19.997229 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caa38c67caff92d65ba0dd559205054eb3fbd8f0aedba0c9f7c9f1ab5da2ead8"} err="failed to get container status \"caa38c67caff92d65ba0dd559205054eb3fbd8f0aedba0c9f7c9f1ab5da2ead8\": rpc error: code = NotFound desc = could not find container \"caa38c67caff92d65ba0dd559205054eb3fbd8f0aedba0c9f7c9f1ab5da2ead8\": container with ID starting with caa38c67caff92d65ba0dd559205054eb3fbd8f0aedba0c9f7c9f1ab5da2ead8 not found: ID does not exist" Jul 09 14:21:20.011230 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:20.011202 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6ff87f966f-qmfd5"] Jul 09 14:21:20.018536 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:20.018511 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6ff87f966f-qmfd5"] Jul 09 14:21:20.985072 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:20.985030 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="036b5417-59df-45e9-8f35-3f1d729b4281" path="/var/lib/kubelet/pods/036b5417-59df-45e9-8f35-3f1d729b4281/volumes" Jul 09 14:21:24.013225 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:24.013182 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-zsj7t" event={"ID":"3defbe4a-fb5a-4671-a840-0ee2b6c8e696","Type":"ContainerStarted","Data":"4b237661e012d2d4e4c0e18778d1b64be4c045f4d5229366199bf07282ae05ba"} Jul 09 14:21:24.013633 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:24.013248 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-zsj7t" Jul 09 14:21:24.030508 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:24.030452 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-zsj7t" podStartSLOduration=1.842003326 podStartE2EDuration="5.030437727s" totalImagesPullingTime="3.188434401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:21:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:24.029081588 +0000 UTC m=+404.152220990" watchObservedRunningTime="2026-07-09 14:21:24.030437727 +0000 UTC m=+404.153577109" Jul 09 14:21:35.019453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:21:35.019416 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-zsj7t" Jul 09 14:22:36.023505 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:36.023465 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:22:43.438528 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:43.438499 2581 generic.go:356] "Generic (PLEG): container finished" podID="34d9a37b-862d-4a2a-90f8-72c2e07cf9f9" containerID="57036640473fab22f8225bcfa0552496b3f55727dd4c1d861eb47fe4ffdc26ad" exitCode=0 Jul 09 14:22:43.438844 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:43.438569 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" event={"ID":"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9","Type":"ContainerDied","Data":"57036640473fab22f8225bcfa0552496b3f55727dd4c1d861eb47fe4ffdc26ad"} Jul 09 14:22:43.439002 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:43.438968 2581 scope.go:117] "RemoveContainer" containerID="57036640473fab22f8225bcfa0552496b3f55727dd4c1d861eb47fe4ffdc26ad" Jul 09 14:22:43.440070 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:43.440044 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:22:43.440177 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:43.440127 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:22:43.440236 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:43.440172 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:22:44.446147 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:44.446110 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" event={"ID":"34d9a37b-862d-4a2a-90f8-72c2e07cf9f9","Type":"ContainerStarted","Data":"ee6cd50209c84709d04ab2129d863071ebe0fb0e3d016852fdd5dd07ec1dbf00"} Jul 09 14:22:44.930128 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:44.930093 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:22:44.930347 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:44.930322 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6sngn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 09 14:22:44.930396 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:44.930379 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" podUID="34d9a37b-862d-4a2a-90f8-72c2e07cf9f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 09 14:22:45.930408 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:45.930369 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6sngn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 09 14:22:45.930828 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:45.930426 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" podUID="34d9a37b-862d-4a2a-90f8-72c2e07cf9f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 09 14:22:46.929825 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:46.929786 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6sngn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 09 14:22:46.930035 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:46.929849 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" podUID="34d9a37b-862d-4a2a-90f8-72c2e07cf9f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 09 14:22:47.930198 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:47.930158 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6sngn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 09 14:22:47.930633 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:47.930235 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" podUID="34d9a37b-862d-4a2a-90f8-72c2e07cf9f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 09 14:22:48.930085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:48.930049 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6sngn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 09 14:22:48.930268 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:48.930119 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" podUID="34d9a37b-862d-4a2a-90f8-72c2e07cf9f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 09 14:22:49.934215 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:49.934187 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:22:49.934659 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:49.934461 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:22:49.935193 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:22:49.935175 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6sngn" Jul 09 14:23:01.702431 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:01.702396 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:23:13.505946 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:13.505903 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:23:19.305382 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:19.305346 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:23:25.002121 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:25.002084 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:23:34.398889 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:34.398809 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:23:40.572525 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.572486 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc"] Jul 09 14:23:40.575921 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.575899 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.580685 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.580658 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rmz94\"" Jul 09 14:23:40.580685 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.580660 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 14:23:40.580876 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.580710 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 14:23:40.580932 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.580880 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 09 14:23:40.586450 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.586427 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc"] Jul 09 14:23:40.623150 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.623111 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.623319 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.623163 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.623319 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.623214 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.623319 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.623244 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.623319 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.623263 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t7ch\" (UniqueName: \"kubernetes.io/projected/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-kube-api-access-4t7ch\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.623484 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.623364 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.724389 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.724361 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.724547 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.724399 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.724547 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.724436 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.724547 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.724461 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.724547 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.724483 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.724547 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.724509 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t7ch\" (UniqueName: \"kubernetes.io/projected/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-kube-api-access-4t7ch\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.724815 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.724780 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.724872 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.724813 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.724872 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.724847 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.726815 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.726793 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.727014 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.726995 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.734319 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.734293 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t7ch\" (UniqueName: \"kubernetes.io/projected/6bdce3eb-29c2-4681-ac86-12fdb98d8d9b-kube-api-access-4t7ch\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc\" (UID: \"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:40.885490 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:40.885452 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:41.031413 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:41.031368 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc"] Jul 09 14:23:41.033944 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:23:41.033915 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bdce3eb_29c2_4681_ac86_12fdb98d8d9b.slice/crio-b908e95e4adcd08475443d08f1f26ea13a248bc4776038d73274bd16752b59b9 WatchSource:0}: Error finding container b908e95e4adcd08475443d08f1f26ea13a248bc4776038d73274bd16752b59b9: Status 404 returned error can't find the container with id b908e95e4adcd08475443d08f1f26ea13a248bc4776038d73274bd16752b59b9 Jul 09 14:23:41.744438 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:41.744404 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" event={"ID":"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b","Type":"ContainerStarted","Data":"b908e95e4adcd08475443d08f1f26ea13a248bc4776038d73274bd16752b59b9"} Jul 09 14:23:42.750943 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:42.750901 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" event={"ID":"6bdce3eb-29c2-4681-ac86-12fdb98d8d9b","Type":"ContainerStarted","Data":"e322d36d5d0f8fc0461ae5fd840dbab25cb63d15f5ca04723b94a3697ac5c6f9"} Jul 09 14:23:42.774139 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:42.774082 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" podStartSLOduration=1.731208254 podStartE2EDuration="2.774067441s" totalImagesPullingTime="1.042859187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:23:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:23:42.771629775 +0000 UTC m=+542.894769175" watchObservedRunningTime="2026-07-09 14:23:42.774067441 +0000 UTC m=+542.897206823" Jul 09 14:23:43.605659 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:43.605617 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:23:50.886255 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:50.886218 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:50.886663 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:50.886271 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:50.898928 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:50.898898 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:23:51.811198 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:23:51.811167 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc" Jul 09 14:24:40.666827 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:24:40.666799 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:24:40.667423 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:24:40.666842 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:25:08.006492 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:25:08.006409 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:25:19.705639 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:25:19.705605 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:25:30.220117 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:25:30.220085 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:25:38.519678 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:25:38.519638 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:25:49.407445 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:25:49.407412 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:26:17.617789 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:26:17.617754 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:26:28.919778 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:26:28.919738 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:27:10.006185 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:27:10.006152 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:27:19.698108 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:27:19.698066 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:27:29.005222 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:27:29.005189 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:27:34.804052 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:27:34.804004 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:27:45.808060 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:27:45.808021 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:27:55.405965 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:27:55.405924 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:28:06.396965 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:28:06.396869 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:28:16.605790 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:28:16.605751 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:28:41.600934 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:28:41.600894 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:28:52.098898 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:28:52.098856 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:29:16.210764 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:29:16.210722 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:29:28.205111 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:29:28.205070 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:29:40.700889 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:29:40.700856 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:29:40.701791 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:29:40.701768 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:29:44.301908 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:29:44.301872 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:29:53.406532 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:29:53.406495 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:30:02.927219 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:30:02.927180 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:30:14.296328 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:30:14.296285 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:30:23.705983 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:30:23.705932 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:32:10.004977 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:32:10.004925 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:32:22.730140 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:32:22.730103 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:32:31.114375 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:32:31.114288 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:32:42.803214 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:32:42.803173 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:32:53.502763 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:32:53.502719 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:33:04.304054 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:33:04.304021 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:33:13.717615 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:33:13.717581 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:33:24.860736 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:33:24.860697 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:33:35.400782 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:33:35.400740 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:33:44.112043 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:33:44.112004 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:33:55.730326 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:33:55.730281 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:34:20.109369 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:34:20.109286 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:34:28.205468 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:34:28.205427 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:34:40.595622 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:34:40.595579 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:34:40.734063 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:34:40.734035 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:34:40.735663 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:34:40.735640 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:34:52.097925 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:34:52.097888 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:35:07.111627 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:35:07.111588 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:35:21.210582 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:35:21.210546 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:35:30.149464 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:35:30.149425 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:35:45.095996 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:35:45.095958 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:35:55.610176 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:35:55.610134 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:36:06.400795 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:36:06.400756 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:36:16.001856 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:36:16.001818 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:36:25.505510 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:36:25.505469 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:36:34.999549 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:36:34.999511 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:36:47.358227 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:36:47.358189 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:36:55.401938 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:36:55.401900 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:37:07.708562 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:07.708480 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:37:17.543254 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:17.543215 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:37:29.098451 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:29.098415 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:37:39.627234 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.627200 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm"] Jul 09 14:37:39.631203 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.631182 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.636066 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.636040 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5d40eff7-openshift-default-dockercfg-66r2v\"" Jul 09 14:37:39.673790 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.673755 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm"] Jul 09 14:37:39.749758 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.749722 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt8dm\" (UniqueName: \"kubernetes.io/projected/ea26d458-b675-4600-8ac5-78568c7ceda5-kube-api-access-nt8dm\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.749932 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.749775 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-credential-socket\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.749932 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.749806 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-data\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.749932 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.749872 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-podinfo\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.750123 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.750012 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-workload-socket\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.750123 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.750049 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-token\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.750123 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.750074 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea26d458-b675-4600-8ac5-78568c7ceda5-istiod-ca-cert\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.750226 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.750128 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-workload-certs\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.750226 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.750165 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-envoy\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.850859 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.850807 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt8dm\" (UniqueName: \"kubernetes.io/projected/ea26d458-b675-4600-8ac5-78568c7ceda5-kube-api-access-nt8dm\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.850859 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.850866 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-credential-socket\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.851149 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.850887 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-data\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.851149 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.850906 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-podinfo\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.851149 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.850930 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-workload-socket\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.851149 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.850946 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-token\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.851149 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.850992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea26d458-b675-4600-8ac5-78568c7ceda5-istiod-ca-cert\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.851399 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.851188 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-workload-certs\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.851399 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.851240 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-envoy\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.851491 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.851404 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-data\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.851491 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.851457 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-credential-socket\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.851590 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.851508 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-workload-socket\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.851590 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.851539 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-workload-certs\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.851822 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.851802 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea26d458-b675-4600-8ac5-78568c7ceda5-istiod-ca-cert\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.853386 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.853365 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-envoy\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.853731 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.853708 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-podinfo\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.866060 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.866031 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt8dm\" (UniqueName: \"kubernetes.io/projected/ea26d458-b675-4600-8ac5-78568c7ceda5-kube-api-access-nt8dm\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.867144 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.867126 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-token\") pod \"e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:39.942256 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:39.942163 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:40.094453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:40.094429 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm"] Jul 09 14:37:40.096217 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:37:40.096191 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea26d458_b675_4600_8ac5_78568c7ceda5.slice/crio-7970359d7af6df40e03674f713aa0cd8123e4de831c72a8377472b40112dcc6a WatchSource:0}: Error finding container 7970359d7af6df40e03674f713aa0cd8123e4de831c72a8377472b40112dcc6a: Status 404 returned error can't find the container with id 7970359d7af6df40e03674f713aa0cd8123e4de831c72a8377472b40112dcc6a Jul 09 14:37:40.098446 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:40.098412 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:37:40.098530 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:40.098479 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:37:40.098530 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:40.098509 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:37:40.129579 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:40.129550 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" event={"ID":"ea26d458-b675-4600-8ac5-78568c7ceda5","Type":"ContainerStarted","Data":"7970359d7af6df40e03674f713aa0cd8123e4de831c72a8377472b40112dcc6a"} Jul 09 14:37:41.136836 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:41.136791 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" event={"ID":"ea26d458-b675-4600-8ac5-78568c7ceda5","Type":"ContainerStarted","Data":"480353a9a80e621325903942fd05dcbd88b2e9146fc5ffb852540b68a9e4d078"} Jul 09 14:37:41.165807 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:41.165752 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" podStartSLOduration=2.165735824 podStartE2EDuration="2.165735824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:37:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:37:41.164093302 +0000 UTC m=+1381.287232686" watchObservedRunningTime="2026-07-09 14:37:41.165735824 +0000 UTC m=+1381.288875277" Jul 09 14:37:41.943408 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:41.943368 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:41.948174 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:41.948148 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:42.143094 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:42.143064 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:42.144002 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:42.143984 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:43.696763 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:43.696731 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:37:46.056941 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:46.056902 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm"] Jul 09 14:37:46.057431 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:46.057138 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" podUID="ea26d458-b675-4600-8ac5-78568c7ceda5" containerName="istio-proxy" containerID="cri-o://480353a9a80e621325903942fd05dcbd88b2e9146fc5ffb852540b68a9e4d078" gracePeriod=30 Jul 09 14:37:52.610885 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.610846 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:37:52.920306 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.920280 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:52.971929 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.971840 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-podinfo\") pod \"ea26d458-b675-4600-8ac5-78568c7ceda5\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " Jul 09 14:37:52.971929 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.971901 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-workload-certs\") pod \"ea26d458-b675-4600-8ac5-78568c7ceda5\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " Jul 09 14:37:52.972160 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.971936 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nt8dm\" (UniqueName: \"kubernetes.io/projected/ea26d458-b675-4600-8ac5-78568c7ceda5-kube-api-access-nt8dm\") pod \"ea26d458-b675-4600-8ac5-78568c7ceda5\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " Jul 09 14:37:52.972160 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.972089 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-envoy\") pod \"ea26d458-b675-4600-8ac5-78568c7ceda5\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " Jul 09 14:37:52.972269 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.972185 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-data\") pod \"ea26d458-b675-4600-8ac5-78568c7ceda5\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " Jul 09 14:37:52.972269 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.972203 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ea26d458-b675-4600-8ac5-78568c7ceda5" (UID: "ea26d458-b675-4600-8ac5-78568c7ceda5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:37:52.972269 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.972224 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-workload-socket\") pod \"ea26d458-b675-4600-8ac5-78568c7ceda5\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " Jul 09 14:37:52.972269 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.972260 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-token\") pod \"ea26d458-b675-4600-8ac5-78568c7ceda5\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " Jul 09 14:37:52.972470 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.972290 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea26d458-b675-4600-8ac5-78568c7ceda5-istiod-ca-cert\") pod \"ea26d458-b675-4600-8ac5-78568c7ceda5\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " Jul 09 14:37:52.972470 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.972333 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-credential-socket\") pod \"ea26d458-b675-4600-8ac5-78568c7ceda5\" (UID: \"ea26d458-b675-4600-8ac5-78568c7ceda5\") " Jul 09 14:37:52.972675 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.972649 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-workload-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:37:52.972804 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.972702 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ea26d458-b675-4600-8ac5-78568c7ceda5" (UID: "ea26d458-b675-4600-8ac5-78568c7ceda5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:37:52.972904 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.972880 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ea26d458-b675-4600-8ac5-78568c7ceda5" (UID: "ea26d458-b675-4600-8ac5-78568c7ceda5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:37:52.973540 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.973508 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-data" (OuterVolumeSpecName: "istio-data") pod "ea26d458-b675-4600-8ac5-78568c7ceda5" (UID: "ea26d458-b675-4600-8ac5-78568c7ceda5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:37:52.974485 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.974428 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea26d458-b675-4600-8ac5-78568c7ceda5-kube-api-access-nt8dm" (OuterVolumeSpecName: "kube-api-access-nt8dm") pod "ea26d458-b675-4600-8ac5-78568c7ceda5" (UID: "ea26d458-b675-4600-8ac5-78568c7ceda5"). InnerVolumeSpecName "kube-api-access-nt8dm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:37:52.974626 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.974449 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ea26d458-b675-4600-8ac5-78568c7ceda5" (UID: "ea26d458-b675-4600-8ac5-78568c7ceda5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:37:52.974678 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.974617 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea26d458-b675-4600-8ac5-78568c7ceda5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ea26d458-b675-4600-8ac5-78568c7ceda5" (UID: "ea26d458-b675-4600-8ac5-78568c7ceda5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:37:52.974892 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.974870 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ea26d458-b675-4600-8ac5-78568c7ceda5" (UID: "ea26d458-b675-4600-8ac5-78568c7ceda5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:37:52.976168 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:52.976149 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-token" (OuterVolumeSpecName: "istio-token") pod "ea26d458-b675-4600-8ac5-78568c7ceda5" (UID: "ea26d458-b675-4600-8ac5-78568c7ceda5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:37:53.074141 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.073985 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nt8dm\" (UniqueName: \"kubernetes.io/projected/ea26d458-b675-4600-8ac5-78568c7ceda5-kube-api-access-nt8dm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:37:53.074141 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.074024 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-envoy\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:37:53.074141 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.074037 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-data\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:37:53.074141 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.074050 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-workload-socket\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:37:53.074141 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.074064 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-token\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:37:53.074141 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.074076 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea26d458-b675-4600-8ac5-78568c7ceda5-istiod-ca-cert\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:37:53.074141 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.074089 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea26d458-b675-4600-8ac5-78568c7ceda5-credential-socket\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:37:53.074141 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.074101 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea26d458-b675-4600-8ac5-78568c7ceda5-istio-podinfo\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:37:53.210514 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.210479 2581 generic.go:356] "Generic (PLEG): container finished" podID="ea26d458-b675-4600-8ac5-78568c7ceda5" containerID="480353a9a80e621325903942fd05dcbd88b2e9146fc5ffb852540b68a9e4d078" exitCode=0 Jul 09 14:37:53.210683 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.210559 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" Jul 09 14:37:53.210683 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.210595 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" event={"ID":"ea26d458-b675-4600-8ac5-78568c7ceda5","Type":"ContainerDied","Data":"480353a9a80e621325903942fd05dcbd88b2e9146fc5ffb852540b68a9e4d078"} Jul 09 14:37:53.210683 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.210656 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm" event={"ID":"ea26d458-b675-4600-8ac5-78568c7ceda5","Type":"ContainerDied","Data":"7970359d7af6df40e03674f713aa0cd8123e4de831c72a8377472b40112dcc6a"} Jul 09 14:37:53.210683 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.210679 2581 scope.go:117] "RemoveContainer" containerID="480353a9a80e621325903942fd05dcbd88b2e9146fc5ffb852540b68a9e4d078" Jul 09 14:37:53.220024 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.220004 2581 scope.go:117] "RemoveContainer" containerID="480353a9a80e621325903942fd05dcbd88b2e9146fc5ffb852540b68a9e4d078" Jul 09 14:37:53.220316 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:37:53.220291 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"480353a9a80e621325903942fd05dcbd88b2e9146fc5ffb852540b68a9e4d078\": container with ID starting with 480353a9a80e621325903942fd05dcbd88b2e9146fc5ffb852540b68a9e4d078 not found: ID does not exist" containerID="480353a9a80e621325903942fd05dcbd88b2e9146fc5ffb852540b68a9e4d078" Jul 09 14:37:53.220422 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.220323 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"480353a9a80e621325903942fd05dcbd88b2e9146fc5ffb852540b68a9e4d078"} err="failed to get container status \"480353a9a80e621325903942fd05dcbd88b2e9146fc5ffb852540b68a9e4d078\": rpc error: code = NotFound desc = could not find container \"480353a9a80e621325903942fd05dcbd88b2e9146fc5ffb852540b68a9e4d078\": container with ID starting with 480353a9a80e621325903942fd05dcbd88b2e9146fc5ffb852540b68a9e4d078 not found: ID does not exist" Jul 09 14:37:53.235511 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.235448 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm"] Jul 09 14:37:53.240469 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:53.240445 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5d40eff7-openshift-default-9c4555cdf-gnsrm"] Jul 09 14:37:54.978332 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:37:54.978297 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea26d458-b675-4600-8ac5-78568c7ceda5" path="/var/lib/kubelet/pods/ea26d458-b675-4600-8ac5-78568c7ceda5/volumes" Jul 09 14:39:40.768369 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:39:40.768339 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:39:40.770703 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:39:40.770678 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:39:50.369135 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:39:50.369095 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:39:58.033658 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:39:58.033623 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:40:05.050313 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.050229 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj"] Jul 09 14:40:05.050667 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.050580 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea26d458-b675-4600-8ac5-78568c7ceda5" containerName="istio-proxy" Jul 09 14:40:05.050667 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.050591 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea26d458-b675-4600-8ac5-78568c7ceda5" containerName="istio-proxy" Jul 09 14:40:05.050667 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.050667 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea26d458-b675-4600-8ac5-78568c7ceda5" containerName="istio-proxy" Jul 09 14:40:05.053733 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.053715 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.057048 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.057022 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a0d9b122-openshift-default-dockercfg-sd494\"" Jul 09 14:40:05.087787 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.087754 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj"] Jul 09 14:40:05.091105 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.091075 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-workload-socket\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.091302 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.091136 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk5wd\" (UniqueName: \"kubernetes.io/projected/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-kube-api-access-pk5wd\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.091302 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.091181 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-workload-certs\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.091302 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.091212 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-envoy\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.091302 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.091242 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-podinfo\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.091302 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.091290 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-data\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.091576 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.091403 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-credential-socket\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.091576 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.091452 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-token\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.091576 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.091477 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istiod-ca-cert\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.192766 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.192729 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-credential-socket\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.192940 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.192781 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-token\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.192940 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.192816 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istiod-ca-cert\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.192940 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.192866 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-workload-socket\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.192940 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.192912 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk5wd\" (UniqueName: \"kubernetes.io/projected/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-kube-api-access-pk5wd\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.193219 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.192945 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-workload-certs\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.193219 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.193009 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-envoy\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.193219 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.193044 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-podinfo\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.193219 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.193079 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-data\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.193219 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.193198 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-credential-socket\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.193474 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.193383 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-workload-certs\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.193548 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.193527 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-data\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.193596 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.193554 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-workload-socket\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.194028 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.194008 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istiod-ca-cert\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.195431 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.195398 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-envoy\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.195853 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.195834 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-podinfo\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.204240 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.204216 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk5wd\" (UniqueName: \"kubernetes.io/projected/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-kube-api-access-pk5wd\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.206263 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.206243 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-token\") pod \"e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.363945 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.363907 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:05.508045 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.508020 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj"] Jul 09 14:40:05.510325 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:40:05.510300 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc0cba8_0d97_483d_8d8f_57d692f5f2d4.slice/crio-d1322ee7c8ba5d15b92ed6a5166e7967c48d43508a7703e7668a5d4fed698775 WatchSource:0}: Error finding container d1322ee7c8ba5d15b92ed6a5166e7967c48d43508a7703e7668a5d4fed698775: Status 404 returned error can't find the container with id d1322ee7c8ba5d15b92ed6a5166e7967c48d43508a7703e7668a5d4fed698775 Jul 09 14:40:05.512620 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.512585 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:40:05.512720 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.512658 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:40:05.512720 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.512688 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:40:05.910943 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.910905 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" event={"ID":"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4","Type":"ContainerStarted","Data":"2a099686d04007783c6ce9367e25bdfb0da6ed6e2e544318ce069fb0fac9527d"} Jul 09 14:40:05.911143 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.910981 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" event={"ID":"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4","Type":"ContainerStarted","Data":"d1322ee7c8ba5d15b92ed6a5166e7967c48d43508a7703e7668a5d4fed698775"} Jul 09 14:40:05.939006 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:05.938928 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" podStartSLOduration=0.938911381 podStartE2EDuration="938.911381ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:40:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:40:05.937508035 +0000 UTC m=+1526.060647419" watchObservedRunningTime="2026-07-09 14:40:05.938911381 +0000 UTC m=+1526.062050764" Jul 09 14:40:06.364942 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:06.364900 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:06.369971 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:06.369927 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:06.705585 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:06.705502 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:40:06.917428 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:06.917394 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:06.918550 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:06.918533 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:16.808069 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:16.808035 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:40:20.999322 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:20.999288 2581 generic.go:356] "Generic (PLEG): container finished" podID="6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" containerID="2a099686d04007783c6ce9367e25bdfb0da6ed6e2e544318ce069fb0fac9527d" exitCode=0 Jul 09 14:40:20.999714 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:20.999364 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" event={"ID":"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4","Type":"ContainerDied","Data":"2a099686d04007783c6ce9367e25bdfb0da6ed6e2e544318ce069fb0fac9527d"} Jul 09 14:40:20.999776 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:20.999762 2581 scope.go:117] "RemoveContainer" containerID="2a099686d04007783c6ce9367e25bdfb0da6ed6e2e544318ce069fb0fac9527d" Jul 09 14:40:21.000802 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:21.000773 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:40:21.000912 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:21.000845 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:40:21.000912 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:21.000876 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:40:21.917702 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:21.917667 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:22.007009 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:22.006966 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" event={"ID":"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4","Type":"ContainerStarted","Data":"d9988a80482b72a229da5090cadb632400b49d11420ac926f424e93cae585a1c"} Jul 09 14:40:22.364883 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:22.364846 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:22.366082 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:22.366056 2581 patch_prober.go:28] interesting pod/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 09 14:40:22.366207 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:22.366123 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" podUID="6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 09 14:40:23.364789 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:23.364748 2581 patch_prober.go:28] interesting pod/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 09 14:40:23.365200 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:23.364804 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" podUID="6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 09 14:40:23.919151 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:23.919117 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:40:24.364266 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:24.364222 2581 patch_prober.go:28] interesting pod/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 09 14:40:24.364465 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:24.364284 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" podUID="6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 09 14:40:25.365056 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:25.365017 2581 patch_prober.go:28] interesting pod/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 09 14:40:25.365453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:25.365075 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" podUID="6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 09 14:40:26.368559 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:26.368528 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:26.368926 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:26.368789 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:26.369581 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:26.369559 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:26.440805 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:26.440770 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj"] Jul 09 14:40:27.033626 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:27.033593 2581 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" secret="" err="secret \"e2e-mt-a0d9b122-openshift-default-dockercfg-sd494\" not found" Jul 09 14:40:28.038878 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:28.038839 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" podUID="6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" containerName="istio-proxy" containerID="cri-o://d9988a80482b72a229da5090cadb632400b49d11420ac926f424e93cae585a1c" gracePeriod=30 Jul 09 14:40:33.298581 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.298512 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:33.372183 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.372150 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istiod-ca-cert\") pod \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " Jul 09 14:40:33.372389 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.372192 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk5wd\" (UniqueName: \"kubernetes.io/projected/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-kube-api-access-pk5wd\") pod \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " Jul 09 14:40:33.372389 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.372220 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-workload-certs\") pod \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " Jul 09 14:40:33.372389 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.372255 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-credential-socket\") pod \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " Jul 09 14:40:33.372389 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.372294 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-data\") pod \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " Jul 09 14:40:33.372389 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.372368 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-token\") pod \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " Jul 09 14:40:33.372673 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.372394 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-envoy\") pod \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " Jul 09 14:40:33.372673 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.372427 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-podinfo\") pod \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " Jul 09 14:40:33.372673 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.372461 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-workload-socket\") pod \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\" (UID: \"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4\") " Jul 09 14:40:33.372673 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.372516 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" (UID: "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:40:33.372673 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.372627 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" (UID: "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:40:33.372936 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.372840 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" (UID: "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:40:33.372936 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.372883 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-workload-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:40:33.372936 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.372902 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-credential-socket\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:40:33.373213 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.373193 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" (UID: "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:40:33.373339 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.373319 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-data" (OuterVolumeSpecName: "istio-data") pod "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" (UID: "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:40:33.374636 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.374617 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" (UID: "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:40:33.374768 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.374751 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-kube-api-access-pk5wd" (OuterVolumeSpecName: "kube-api-access-pk5wd") pod "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" (UID: "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4"). InnerVolumeSpecName "kube-api-access-pk5wd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:40:33.375029 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.375007 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-token" (OuterVolumeSpecName: "istio-token") pod "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" (UID: "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:40:33.375083 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.375036 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" (UID: "6cc0cba8-0d97-483d-8d8f-57d692f5f2d4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:40:33.473691 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.473652 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk5wd\" (UniqueName: \"kubernetes.io/projected/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-kube-api-access-pk5wd\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:40:33.473691 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.473684 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-data\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:40:33.473923 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.473707 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-token\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:40:33.473923 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.473720 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-envoy\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:40:33.473923 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.473732 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istio-podinfo\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:40:33.473923 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.473743 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-workload-socket\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:40:33.473923 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:33.473753 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4-istiod-ca-cert\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:40:34.072473 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:34.072438 2581 generic.go:356] "Generic (PLEG): container finished" podID="6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" containerID="d9988a80482b72a229da5090cadb632400b49d11420ac926f424e93cae585a1c" exitCode=0 Jul 09 14:40:34.072656 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:34.072487 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" Jul 09 14:40:34.072656 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:34.072517 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" event={"ID":"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4","Type":"ContainerDied","Data":"d9988a80482b72a229da5090cadb632400b49d11420ac926f424e93cae585a1c"} Jul 09 14:40:34.072656 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:34.072571 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj" event={"ID":"6cc0cba8-0d97-483d-8d8f-57d692f5f2d4","Type":"ContainerDied","Data":"d1322ee7c8ba5d15b92ed6a5166e7967c48d43508a7703e7668a5d4fed698775"} Jul 09 14:40:34.072656 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:34.072592 2581 scope.go:117] "RemoveContainer" containerID="d9988a80482b72a229da5090cadb632400b49d11420ac926f424e93cae585a1c" Jul 09 14:40:34.082405 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:34.082381 2581 scope.go:117] "RemoveContainer" containerID="2a099686d04007783c6ce9367e25bdfb0da6ed6e2e544318ce069fb0fac9527d" Jul 09 14:40:34.090478 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:34.090461 2581 scope.go:117] "RemoveContainer" containerID="d9988a80482b72a229da5090cadb632400b49d11420ac926f424e93cae585a1c" Jul 09 14:40:34.090720 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:40:34.090697 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9988a80482b72a229da5090cadb632400b49d11420ac926f424e93cae585a1c\": container with ID starting with d9988a80482b72a229da5090cadb632400b49d11420ac926f424e93cae585a1c not found: ID does not exist" containerID="d9988a80482b72a229da5090cadb632400b49d11420ac926f424e93cae585a1c" Jul 09 14:40:34.090787 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:34.090733 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9988a80482b72a229da5090cadb632400b49d11420ac926f424e93cae585a1c"} err="failed to get container status \"d9988a80482b72a229da5090cadb632400b49d11420ac926f424e93cae585a1c\": rpc error: code = NotFound desc = could not find container \"d9988a80482b72a229da5090cadb632400b49d11420ac926f424e93cae585a1c\": container with ID starting with d9988a80482b72a229da5090cadb632400b49d11420ac926f424e93cae585a1c not found: ID does not exist" Jul 09 14:40:34.090787 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:34.090758 2581 scope.go:117] "RemoveContainer" containerID="2a099686d04007783c6ce9367e25bdfb0da6ed6e2e544318ce069fb0fac9527d" Jul 09 14:40:34.091025 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:40:34.091003 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a099686d04007783c6ce9367e25bdfb0da6ed6e2e544318ce069fb0fac9527d\": container with ID starting with 2a099686d04007783c6ce9367e25bdfb0da6ed6e2e544318ce069fb0fac9527d not found: ID does not exist" containerID="2a099686d04007783c6ce9367e25bdfb0da6ed6e2e544318ce069fb0fac9527d" Jul 09 14:40:34.091076 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:34.091036 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a099686d04007783c6ce9367e25bdfb0da6ed6e2e544318ce069fb0fac9527d"} err="failed to get container status \"2a099686d04007783c6ce9367e25bdfb0da6ed6e2e544318ce069fb0fac9527d\": rpc error: code = NotFound desc = could not find container \"2a099686d04007783c6ce9367e25bdfb0da6ed6e2e544318ce069fb0fac9527d\": container with ID starting with 2a099686d04007783c6ce9367e25bdfb0da6ed6e2e544318ce069fb0fac9527d not found: ID does not exist" Jul 09 14:40:34.107718 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:34.107692 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj"] Jul 09 14:40:34.114592 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:34.114567 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a0d9b122-openshift-default-b4759cfd7-6t5fj"] Jul 09 14:40:34.978214 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:34.978179 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" path="/var/lib/kubelet/pods/6cc0cba8-0d97-483d-8d8f-57d692f5f2d4/volumes" Jul 09 14:40:55.201195 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:40:55.201162 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:41:02.576474 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.576431 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc"] Jul 09 14:41:02.576849 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.576792 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" containerName="istio-proxy" Jul 09 14:41:02.576849 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.576802 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" containerName="istio-proxy" Jul 09 14:41:02.576849 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.576820 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" containerName="istio-proxy" Jul 09 14:41:02.576849 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.576826 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" containerName="istio-proxy" Jul 09 14:41:02.577021 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.576880 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" containerName="istio-proxy" Jul 09 14:41:02.581022 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.581004 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.585549 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.585522 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2a4ea46b\"/\"default-dockercfg-vg8dn\"" Jul 09 14:41:02.585684 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.585599 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2a4ea46b\"/\"kube-root-ca.crt\"" Jul 09 14:41:02.585684 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.585652 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2a4ea46b\"/\"openshift-service-ca.crt\"" Jul 09 14:41:02.585779 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.585746 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2a4ea46b\"/\"e2e-lifecycle-model-2a4ea46b-kserve-self-signed-certs\"" Jul 09 14:41:02.591479 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.591455 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc"] Jul 09 14:41:02.744214 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.744179 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/758b1ef6-272a-4078-81df-2a9fbe175feb-tls-certs\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.744398 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.744247 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-model-cache\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.744398 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.744279 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-home\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.744398 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.744308 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-tmp-dir\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.744398 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.744361 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-dshm\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.744543 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.744415 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn9ll\" (UniqueName: \"kubernetes.io/projected/758b1ef6-272a-4078-81df-2a9fbe175feb-kube-api-access-qn9ll\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.845515 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.845481 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/758b1ef6-272a-4078-81df-2a9fbe175feb-tls-certs\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.845673 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.845550 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-model-cache\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.845673 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.845581 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-home\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.845673 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.845624 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-tmp-dir\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.845673 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.845663 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-dshm\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.845875 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.845724 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn9ll\" (UniqueName: \"kubernetes.io/projected/758b1ef6-272a-4078-81df-2a9fbe175feb-kube-api-access-qn9ll\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.846020 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.845987 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-model-cache\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.846127 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.846063 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-home\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.846127 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.846107 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-tmp-dir\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.848063 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.848046 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-dshm\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.848157 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.848142 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/758b1ef6-272a-4078-81df-2a9fbe175feb-tls-certs\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.854046 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.854023 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn9ll\" (UniqueName: \"kubernetes.io/projected/758b1ef6-272a-4078-81df-2a9fbe175feb-kube-api-access-qn9ll\") pod \"e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:02.891929 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:02.891892 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:03.030640 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:03.030611 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc"] Jul 09 14:41:03.225975 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:03.225911 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" event={"ID":"758b1ef6-272a-4078-81df-2a9fbe175feb","Type":"ContainerStarted","Data":"d4a6b0b8296d8b1311f9e97f3c29c186506308ef0a043233d22a9cdf3fe6fb1d"} Jul 09 14:41:03.226143 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:03.225985 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" event={"ID":"758b1ef6-272a-4078-81df-2a9fbe175feb","Type":"ContainerStarted","Data":"d16c120144a831b4fac386eafc13ce9a7e9f45178957474cfa3154e69c496feb"} Jul 09 14:41:03.245844 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:03.245792 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" podStartSLOduration=1.245774896 podStartE2EDuration="1.245774896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:41:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:41:03.24317339 +0000 UTC m=+1583.366312773" watchObservedRunningTime="2026-07-09 14:41:03.245774896 +0000 UTC m=+1583.368914278" Jul 09 14:41:06.200396 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:06.200362 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:41:12.892316 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:12.892278 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:12.892739 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:12.892539 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:12.905022 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:12.904998 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:13.289001 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:13.288897 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:18.804028 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:18.803991 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:41:32.098816 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:32.098729 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:41:32.108279 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:32.108247 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc"] Jul 09 14:41:47.128324 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.128281 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" podUID="758b1ef6-272a-4078-81df-2a9fbe175feb" containerName="main" containerID="cri-o://d4a6b0b8296d8b1311f9e97f3c29c186506308ef0a043233d22a9cdf3fe6fb1d" gracePeriod=45 Jul 09 14:41:47.384714 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.384646 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:47.459120 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.459084 2581 generic.go:356] "Generic (PLEG): container finished" podID="758b1ef6-272a-4078-81df-2a9fbe175feb" containerID="d4a6b0b8296d8b1311f9e97f3c29c186506308ef0a043233d22a9cdf3fe6fb1d" exitCode=0 Jul 09 14:41:47.459312 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.459132 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" Jul 09 14:41:47.459312 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.459167 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" event={"ID":"758b1ef6-272a-4078-81df-2a9fbe175feb","Type":"ContainerDied","Data":"d4a6b0b8296d8b1311f9e97f3c29c186506308ef0a043233d22a9cdf3fe6fb1d"} Jul 09 14:41:47.459312 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.459206 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc" event={"ID":"758b1ef6-272a-4078-81df-2a9fbe175feb","Type":"ContainerDied","Data":"d16c120144a831b4fac386eafc13ce9a7e9f45178957474cfa3154e69c496feb"} Jul 09 14:41:47.459312 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.459223 2581 scope.go:117] "RemoveContainer" containerID="d4a6b0b8296d8b1311f9e97f3c29c186506308ef0a043233d22a9cdf3fe6fb1d" Jul 09 14:41:47.467788 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.467767 2581 scope.go:117] "RemoveContainer" containerID="d4a6b0b8296d8b1311f9e97f3c29c186506308ef0a043233d22a9cdf3fe6fb1d" Jul 09 14:41:47.468061 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:41:47.468042 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4a6b0b8296d8b1311f9e97f3c29c186506308ef0a043233d22a9cdf3fe6fb1d\": container with ID starting with d4a6b0b8296d8b1311f9e97f3c29c186506308ef0a043233d22a9cdf3fe6fb1d not found: ID does not exist" containerID="d4a6b0b8296d8b1311f9e97f3c29c186506308ef0a043233d22a9cdf3fe6fb1d" Jul 09 14:41:47.468138 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.468074 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4a6b0b8296d8b1311f9e97f3c29c186506308ef0a043233d22a9cdf3fe6fb1d"} err="failed to get container status \"d4a6b0b8296d8b1311f9e97f3c29c186506308ef0a043233d22a9cdf3fe6fb1d\": rpc error: code = NotFound desc = could not find container \"d4a6b0b8296d8b1311f9e97f3c29c186506308ef0a043233d22a9cdf3fe6fb1d\": container with ID starting with d4a6b0b8296d8b1311f9e97f3c29c186506308ef0a043233d22a9cdf3fe6fb1d not found: ID does not exist" Jul 09 14:41:47.559556 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.559516 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/758b1ef6-272a-4078-81df-2a9fbe175feb-tls-certs\") pod \"758b1ef6-272a-4078-81df-2a9fbe175feb\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " Jul 09 14:41:47.559556 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.559562 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-tmp-dir\") pod \"758b1ef6-272a-4078-81df-2a9fbe175feb\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " Jul 09 14:41:47.559813 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.559651 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-dshm\") pod \"758b1ef6-272a-4078-81df-2a9fbe175feb\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " Jul 09 14:41:47.559813 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.559692 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qn9ll\" (UniqueName: \"kubernetes.io/projected/758b1ef6-272a-4078-81df-2a9fbe175feb-kube-api-access-qn9ll\") pod \"758b1ef6-272a-4078-81df-2a9fbe175feb\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " Jul 09 14:41:47.559813 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.559737 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-model-cache\") pod \"758b1ef6-272a-4078-81df-2a9fbe175feb\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " Jul 09 14:41:47.559813 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.559758 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-home\") pod \"758b1ef6-272a-4078-81df-2a9fbe175feb\" (UID: \"758b1ef6-272a-4078-81df-2a9fbe175feb\") " Jul 09 14:41:47.560054 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.559901 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "758b1ef6-272a-4078-81df-2a9fbe175feb" (UID: "758b1ef6-272a-4078-81df-2a9fbe175feb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:41:47.560054 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.560012 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-model-cache" (OuterVolumeSpecName: "model-cache") pod "758b1ef6-272a-4078-81df-2a9fbe175feb" (UID: "758b1ef6-272a-4078-81df-2a9fbe175feb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:41:47.560168 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.560072 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-home" (OuterVolumeSpecName: "home") pod "758b1ef6-272a-4078-81df-2a9fbe175feb" (UID: "758b1ef6-272a-4078-81df-2a9fbe175feb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:41:47.560168 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.560151 2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:41:47.560245 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.560172 2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:41:47.562028 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.562002 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-dshm" (OuterVolumeSpecName: "dshm") pod "758b1ef6-272a-4078-81df-2a9fbe175feb" (UID: "758b1ef6-272a-4078-81df-2a9fbe175feb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:41:47.562148 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.562048 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/758b1ef6-272a-4078-81df-2a9fbe175feb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "758b1ef6-272a-4078-81df-2a9fbe175feb" (UID: "758b1ef6-272a-4078-81df-2a9fbe175feb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:41:47.562148 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.562088 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/758b1ef6-272a-4078-81df-2a9fbe175feb-kube-api-access-qn9ll" (OuterVolumeSpecName: "kube-api-access-qn9ll") pod "758b1ef6-272a-4078-81df-2a9fbe175feb" (UID: "758b1ef6-272a-4078-81df-2a9fbe175feb"). InnerVolumeSpecName "kube-api-access-qn9ll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:41:47.660923 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.660833 2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:41:47.660923 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.660864 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qn9ll\" (UniqueName: \"kubernetes.io/projected/758b1ef6-272a-4078-81df-2a9fbe175feb-kube-api-access-qn9ll\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:41:47.660923 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.660876 2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/758b1ef6-272a-4078-81df-2a9fbe175feb-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:41:47.660923 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.660884 2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/758b1ef6-272a-4078-81df-2a9fbe175feb-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:41:47.782875 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.782843 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc"] Jul 09 14:41:47.788422 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:47.788377 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-2a4ea46b/e2e-lifecycle-model-2a4ea46b-kserve-7c8f95fd4c-nrbqc"] Jul 09 14:41:48.982937 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:41:48.982892 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="758b1ef6-272a-4078-81df-2a9fbe175feb" path="/var/lib/kubelet/pods/758b1ef6-272a-4078-81df-2a9fbe175feb/volumes" Jul 09 14:42:11.990227 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:42:11.990189 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:42:25.928923 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:42:25.928890 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:42:35.612995 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:42:35.612934 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:43:10.289855 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.289771 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5"] Jul 09 14:43:10.290348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.290221 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="758b1ef6-272a-4078-81df-2a9fbe175feb" containerName="main" Jul 09 14:43:10.290348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.290236 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="758b1ef6-272a-4078-81df-2a9fbe175feb" containerName="main" Jul 09 14:43:10.290348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.290312 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cc0cba8-0d97-483d-8d8f-57d692f5f2d4" containerName="istio-proxy" Jul 09 14:43:10.290348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.290324 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="758b1ef6-272a-4078-81df-2a9fbe175feb" containerName="main" Jul 09 14:43:10.294522 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.294504 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.297492 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.297477 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-d2daa1-openshift-default-dockercfg-2blrv\"" Jul 09 14:43:10.304254 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.304224 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5"] Jul 09 14:43:10.405571 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.405531 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa5086bc-fed4-481c-b531-7915e228de62-istio-token\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.405571 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.405576 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa5086bc-fed4-481c-b531-7915e228de62-istiod-ca-cert\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.405782 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.405612 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-credential-socket\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.405782 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.405634 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa5086bc-fed4-481c-b531-7915e228de62-istio-podinfo\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.405782 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.405670 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-istio-envoy\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.405782 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.405760 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-workload-socket\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.405913 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.405794 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-workload-certs\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.405913 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.405812 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-istio-data\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.405913 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.405843 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nhmh\" (UniqueName: \"kubernetes.io/projected/aa5086bc-fed4-481c-b531-7915e228de62-kube-api-access-8nhmh\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.507051 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.507016 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa5086bc-fed4-481c-b531-7915e228de62-istio-token\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.507222 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.507056 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa5086bc-fed4-481c-b531-7915e228de62-istiod-ca-cert\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.507222 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.507087 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-credential-socket\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.507222 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.507125 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa5086bc-fed4-481c-b531-7915e228de62-istio-podinfo\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.507222 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.507165 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-istio-envoy\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.507446 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.507229 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-workload-socket\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.507446 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.507253 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-workload-certs\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.507446 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.507271 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-istio-data\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.507446 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.507291 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nhmh\" (UniqueName: \"kubernetes.io/projected/aa5086bc-fed4-481c-b531-7915e228de62-kube-api-access-8nhmh\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.507746 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.507708 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-istio-data\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.507885 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.507764 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-credential-socket\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.507885 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.507843 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-workload-certs\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.508041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.507879 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa5086bc-fed4-481c-b531-7915e228de62-istiod-ca-cert\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.508041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.507881 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-workload-socket\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.509560 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.509531 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-istio-envoy\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.509773 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.509752 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa5086bc-fed4-481c-b531-7915e228de62-istio-podinfo\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.517092 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.517056 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa5086bc-fed4-481c-b531-7915e228de62-istio-token\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.517092 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.517082 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nhmh\" (UniqueName: \"kubernetes.io/projected/aa5086bc-fed4-481c-b531-7915e228de62-kube-api-access-8nhmh\") pod \"e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.606544 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.606512 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:10.749003 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.748974 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5"] Jul 09 14:43:10.751972 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:43:10.751921 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa5086bc_fed4_481c_b531_7915e228de62.slice/crio-1e24ad4098e62fbfc7108c89f6a75bd42dc2d50707e9c67e3642421beb6929b2 WatchSource:0}: Error finding container 1e24ad4098e62fbfc7108c89f6a75bd42dc2d50707e9c67e3642421beb6929b2: Status 404 returned error can't find the container with id 1e24ad4098e62fbfc7108c89f6a75bd42dc2d50707e9c67e3642421beb6929b2 Jul 09 14:43:10.754440 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.754407 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:43:10.754518 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.754480 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:43:10.754518 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.754509 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:43:10.893893 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.893796 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" event={"ID":"aa5086bc-fed4-481c-b531-7915e228de62","Type":"ContainerStarted","Data":"8120816ab9b422f2cfc78f953828ef93eb981309b1829b42519b04e3ae0e5732"} Jul 09 14:43:10.893893 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.893852 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" event={"ID":"aa5086bc-fed4-481c-b531-7915e228de62","Type":"ContainerStarted","Data":"1e24ad4098e62fbfc7108c89f6a75bd42dc2d50707e9c67e3642421beb6929b2"} Jul 09 14:43:10.916304 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:10.916241 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" podStartSLOduration=0.916218985 podStartE2EDuration="916.218985ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:43:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:43:10.914733726 +0000 UTC m=+1711.037873141" watchObservedRunningTime="2026-07-09 14:43:10.916218985 +0000 UTC m=+1711.039358369" Jul 09 14:43:11.607457 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:11.607420 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:11.612531 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:11.612505 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:11.899446 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:11.899353 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:11.900513 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:11.900487 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:19.997600 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:19.997562 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:43:27.941163 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:27.941118 2581 patch_prober.go:28] interesting pod/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 14:43:27.941545 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:27.941193 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" podUID="aa5086bc-fed4-481c-b531-7915e228de62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 14:43:28.992911 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:28.992871 2581 generic.go:356] "Generic (PLEG): container finished" podID="aa5086bc-fed4-481c-b531-7915e228de62" containerID="8120816ab9b422f2cfc78f953828ef93eb981309b1829b42519b04e3ae0e5732" exitCode=0 Jul 09 14:43:28.993376 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:28.992943 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" event={"ID":"aa5086bc-fed4-481c-b531-7915e228de62","Type":"ContainerDied","Data":"8120816ab9b422f2cfc78f953828ef93eb981309b1829b42519b04e3ae0e5732"} Jul 09 14:43:28.993376 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:28.993343 2581 scope.go:117] "RemoveContainer" containerID="8120816ab9b422f2cfc78f953828ef93eb981309b1829b42519b04e3ae0e5732" Jul 09 14:43:28.994386 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:28.994352 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:43:28.994491 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:28.994423 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:43:28.994491 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:28.994452 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:43:30.002465 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:30.002429 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" event={"ID":"aa5086bc-fed4-481c-b531-7915e228de62","Type":"ContainerStarted","Data":"bc13e616a9c85f2623c8e227c45d1250d94e5ac5908fc822fe3de02c94b54384"} Jul 09 14:43:30.607363 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:30.607330 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:30.607555 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:30.607533 2581 patch_prober.go:28] interesting pod/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 09 14:43:30.607612 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:30.607593 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" podUID="aa5086bc-fed4-481c-b531-7915e228de62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 09 14:43:31.607510 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:31.607465 2581 patch_prober.go:28] interesting pod/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 09 14:43:31.607917 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:31.607537 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" podUID="aa5086bc-fed4-481c-b531-7915e228de62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 09 14:43:32.596589 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:32.596546 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:43:32.607490 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:32.607456 2581 patch_prober.go:28] interesting pod/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 09 14:43:32.607669 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:32.607534 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" podUID="aa5086bc-fed4-481c-b531-7915e228de62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 09 14:43:33.606879 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:33.606840 2581 patch_prober.go:28] interesting pod/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 09 14:43:33.607090 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:33.606904 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" podUID="aa5086bc-fed4-481c-b531-7915e228de62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 09 14:43:34.610631 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:34.610599 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:34.611093 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:34.610869 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:34.611614 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:34.611598 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:43:44.999040 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:44.999003 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:43:53.872901 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:53.872864 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb"] Jul 09 14:43:53.876685 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:53.876661 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:53.879774 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:53.879749 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4777cd\"/\"kube-root-ca.crt\"" Jul 09 14:43:53.881325 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:53.881225 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4777cd\"/\"default-dockercfg-rhtrm\"" Jul 09 14:43:53.881325 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:53.881264 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4777cd\"/\"openshift-service-ca.crt\"" Jul 09 14:43:53.881533 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:53.881381 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4777cd\"/\"test-model-4777cd-kserve-self-signed-certs\"" Jul 09 14:43:53.887980 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:53.887924 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb"] Jul 09 14:43:54.017086 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.017049 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgbcm\" (UniqueName: \"kubernetes.io/projected/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-kube-api-access-fgbcm\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.017086 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.017093 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-home\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.017316 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.017111 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-model-cache\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.017316 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.017190 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-tmp-dir\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.017316 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.017286 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-tls-certs\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.017316 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.017312 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-dshm\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.118216 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.118169 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-tls-certs\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.118216 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.118229 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-dshm\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.118489 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.118275 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgbcm\" (UniqueName: \"kubernetes.io/projected/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-kube-api-access-fgbcm\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.118489 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.118299 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-home\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.118489 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.118317 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-model-cache\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.118489 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.118341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-tmp-dir\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.118779 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.118745 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-home\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.118852 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.118764 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-model-cache\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.118852 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.118803 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-tmp-dir\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.120613 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.120586 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-dshm\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.120802 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.120788 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-tls-certs\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.127144 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.127083 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgbcm\" (UniqueName: \"kubernetes.io/projected/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-kube-api-access-fgbcm\") pod \"test-model-4777cd-kserve-f976d9b77-2jsmb\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.186818 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.186783 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:43:54.325095 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:54.325071 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb"] Jul 09 14:43:54.326784 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:43:54.326747 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc711ee7b_23f6_4c76_bf8f_4de23ec6b706.slice/crio-f9a3afe239678dfc8080528d9b09b3622d0ce011506e085bd5191f2e3fa9ac1e WatchSource:0}: Error finding container f9a3afe239678dfc8080528d9b09b3622d0ce011506e085bd5191f2e3fa9ac1e: Status 404 returned error can't find the container with id f9a3afe239678dfc8080528d9b09b3622d0ce011506e085bd5191f2e3fa9ac1e Jul 09 14:43:55.140224 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:55.140181 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" event={"ID":"c711ee7b-23f6-4c76-bf8f-4de23ec6b706","Type":"ContainerStarted","Data":"607d7553f2bfbadaeb93a1bee142785e2310b54256bb9ddff6da9140ff7d80ba"} Jul 09 14:43:55.140577 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:55.140235 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" event={"ID":"c711ee7b-23f6-4c76-bf8f-4de23ec6b706","Type":"ContainerStarted","Data":"f9a3afe239678dfc8080528d9b09b3622d0ce011506e085bd5191f2e3fa9ac1e"} Jul 09 14:43:55.160739 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:55.160685 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" podStartSLOduration=2.160668678 podStartE2EDuration="2.160668678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:43:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:43:55.158415793 +0000 UTC m=+1755.281555196" watchObservedRunningTime="2026-07-09 14:43:55.160668678 +0000 UTC m=+1755.283808060" Jul 09 14:43:57.689520 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:43:57.689482 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:44:04.186970 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:04.186922 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:44:04.187370 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:04.186996 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:44:04.199407 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:04.199379 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:44:05.205272 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:05.205243 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:44:14.896493 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:14.896451 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:44:19.553036 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:19.552997 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb"] Jul 09 14:44:31.805831 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:31.805743 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:44:34.573436 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.573390 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" podUID="c711ee7b-23f6-4c76-bf8f-4de23ec6b706" containerName="main" containerID="cri-o://607d7553f2bfbadaeb93a1bee142785e2310b54256bb9ddff6da9140ff7d80ba" gracePeriod=45 Jul 09 14:44:34.820473 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.820443 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:44:34.917805 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.917768 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-dshm\") pod \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " Jul 09 14:44:34.918024 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.917868 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-model-cache\") pod \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " Jul 09 14:44:34.918024 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.917900 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-tls-certs\") pod \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " Jul 09 14:44:34.918024 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.917939 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgbcm\" (UniqueName: \"kubernetes.io/projected/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-kube-api-access-fgbcm\") pod \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " Jul 09 14:44:34.918024 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.918000 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-home\") pod \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " Jul 09 14:44:34.918261 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.918112 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-tmp-dir\") pod \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\" (UID: \"c711ee7b-23f6-4c76-bf8f-4de23ec6b706\") " Jul 09 14:44:34.918261 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.918176 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-model-cache" (OuterVolumeSpecName: "model-cache") pod "c711ee7b-23f6-4c76-bf8f-4de23ec6b706" (UID: "c711ee7b-23f6-4c76-bf8f-4de23ec6b706"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:44:34.918261 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.918247 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-home" (OuterVolumeSpecName: "home") pod "c711ee7b-23f6-4c76-bf8f-4de23ec6b706" (UID: "c711ee7b-23f6-4c76-bf8f-4de23ec6b706"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:44:34.918416 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.918367 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c711ee7b-23f6-4c76-bf8f-4de23ec6b706" (UID: "c711ee7b-23f6-4c76-bf8f-4de23ec6b706"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:44:34.918515 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.918501 2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:44:34.918572 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.918518 2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:44:34.918572 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.918526 2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:44:34.920194 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.920165 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c711ee7b-23f6-4c76-bf8f-4de23ec6b706" (UID: "c711ee7b-23f6-4c76-bf8f-4de23ec6b706"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:44:34.920194 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.920180 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-dshm" (OuterVolumeSpecName: "dshm") pod "c711ee7b-23f6-4c76-bf8f-4de23ec6b706" (UID: "c711ee7b-23f6-4c76-bf8f-4de23ec6b706"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:44:34.920328 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:34.920273 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-kube-api-access-fgbcm" (OuterVolumeSpecName: "kube-api-access-fgbcm") pod "c711ee7b-23f6-4c76-bf8f-4de23ec6b706" (UID: "c711ee7b-23f6-4c76-bf8f-4de23ec6b706"). InnerVolumeSpecName "kube-api-access-fgbcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:44:35.019727 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:35.019695 2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:44:35.019727 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:35.019726 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgbcm\" (UniqueName: \"kubernetes.io/projected/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-kube-api-access-fgbcm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:44:35.019925 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:35.019739 2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c711ee7b-23f6-4c76-bf8f-4de23ec6b706-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:44:35.353919 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:35.353883 2581 generic.go:356] "Generic (PLEG): container finished" podID="c711ee7b-23f6-4c76-bf8f-4de23ec6b706" containerID="607d7553f2bfbadaeb93a1bee142785e2310b54256bb9ddff6da9140ff7d80ba" exitCode=0 Jul 09 14:44:35.354139 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:35.353944 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" Jul 09 14:44:35.354139 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:35.353997 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" event={"ID":"c711ee7b-23f6-4c76-bf8f-4de23ec6b706","Type":"ContainerDied","Data":"607d7553f2bfbadaeb93a1bee142785e2310b54256bb9ddff6da9140ff7d80ba"} Jul 09 14:44:35.354139 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:35.354050 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb" event={"ID":"c711ee7b-23f6-4c76-bf8f-4de23ec6b706","Type":"ContainerDied","Data":"f9a3afe239678dfc8080528d9b09b3622d0ce011506e085bd5191f2e3fa9ac1e"} Jul 09 14:44:35.354139 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:35.354074 2581 scope.go:117] "RemoveContainer" containerID="607d7553f2bfbadaeb93a1bee142785e2310b54256bb9ddff6da9140ff7d80ba" Jul 09 14:44:35.363221 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:35.363205 2581 scope.go:117] "RemoveContainer" containerID="607d7553f2bfbadaeb93a1bee142785e2310b54256bb9ddff6da9140ff7d80ba" Jul 09 14:44:35.363490 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:44:35.363472 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"607d7553f2bfbadaeb93a1bee142785e2310b54256bb9ddff6da9140ff7d80ba\": container with ID starting with 607d7553f2bfbadaeb93a1bee142785e2310b54256bb9ddff6da9140ff7d80ba not found: ID does not exist" containerID="607d7553f2bfbadaeb93a1bee142785e2310b54256bb9ddff6da9140ff7d80ba" Jul 09 14:44:35.363552 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:35.363499 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"607d7553f2bfbadaeb93a1bee142785e2310b54256bb9ddff6da9140ff7d80ba"} err="failed to get container status \"607d7553f2bfbadaeb93a1bee142785e2310b54256bb9ddff6da9140ff7d80ba\": rpc error: code = NotFound desc = could not find container \"607d7553f2bfbadaeb93a1bee142785e2310b54256bb9ddff6da9140ff7d80ba\": container with ID starting with 607d7553f2bfbadaeb93a1bee142785e2310b54256bb9ddff6da9140ff7d80ba not found: ID does not exist" Jul 09 14:44:35.371308 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:35.371286 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb"] Jul 09 14:44:35.375160 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:35.375139 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-4777cd/test-model-4777cd-kserve-f976d9b77-2jsmb"] Jul 09 14:44:36.977734 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:36.977698 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c711ee7b-23f6-4c76-bf8f-4de23ec6b706" path="/var/lib/kubelet/pods/c711ee7b-23f6-4c76-bf8f-4de23ec6b706/volumes" Jul 09 14:44:40.803927 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:40.803898 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:44:40.808064 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:44:40.808039 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:45:02.931972 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:02.931916 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5"] Jul 09 14:45:02.932563 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:02.932215 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" podUID="aa5086bc-fed4-481c-b531-7915e228de62" containerName="istio-proxy" containerID="cri-o://bc13e616a9c85f2623c8e227c45d1250d94e5ac5908fc822fe3de02c94b54384" gracePeriod=30 Jul 09 14:45:04.613548 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:04.613505 2581 patch_prober.go:28] interesting pod/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 14:45:04.613924 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:04.613599 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" podUID="aa5086bc-fed4-481c-b531-7915e228de62" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:45:07.700590 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:07.700557 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:45:08.180421 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.180384 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:45:08.336157 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336072 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa5086bc-fed4-481c-b531-7915e228de62-istiod-ca-cert\") pod \"aa5086bc-fed4-481c-b531-7915e228de62\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " Jul 09 14:45:08.336157 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336131 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-workload-certs\") pod \"aa5086bc-fed4-481c-b531-7915e228de62\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " Jul 09 14:45:08.336157 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336152 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa5086bc-fed4-481c-b531-7915e228de62-istio-token\") pod \"aa5086bc-fed4-481c-b531-7915e228de62\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " Jul 09 14:45:08.336438 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336171 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nhmh\" (UniqueName: \"kubernetes.io/projected/aa5086bc-fed4-481c-b531-7915e228de62-kube-api-access-8nhmh\") pod \"aa5086bc-fed4-481c-b531-7915e228de62\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " Jul 09 14:45:08.336438 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336196 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa5086bc-fed4-481c-b531-7915e228de62-istio-podinfo\") pod \"aa5086bc-fed4-481c-b531-7915e228de62\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " Jul 09 14:45:08.336438 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336225 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-workload-socket\") pod \"aa5086bc-fed4-481c-b531-7915e228de62\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " Jul 09 14:45:08.336438 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336308 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-credential-socket\") pod \"aa5086bc-fed4-481c-b531-7915e228de62\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " Jul 09 14:45:08.336438 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336363 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-istio-data\") pod \"aa5086bc-fed4-481c-b531-7915e228de62\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " Jul 09 14:45:08.336438 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336396 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-istio-envoy\") pod \"aa5086bc-fed4-481c-b531-7915e228de62\" (UID: \"aa5086bc-fed4-481c-b531-7915e228de62\") " Jul 09 14:45:08.336736 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336546 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aa5086bc-fed4-481c-b531-7915e228de62" (UID: "aa5086bc-fed4-481c-b531-7915e228de62"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:08.336736 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336647 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa5086bc-fed4-481c-b531-7915e228de62-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aa5086bc-fed4-481c-b531-7915e228de62" (UID: "aa5086bc-fed4-481c-b531-7915e228de62"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:45:08.336736 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336671 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aa5086bc-fed4-481c-b531-7915e228de62" (UID: "aa5086bc-fed4-481c-b531-7915e228de62"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:08.336912 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336751 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa5086bc-fed4-481c-b531-7915e228de62-istiod-ca-cert\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:08.336912 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336768 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-workload-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:08.336912 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336782 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-credential-socket\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:08.336912 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.336840 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aa5086bc-fed4-481c-b531-7915e228de62" (UID: "aa5086bc-fed4-481c-b531-7915e228de62"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:08.337680 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.337652 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-istio-data" (OuterVolumeSpecName: "istio-data") pod "aa5086bc-fed4-481c-b531-7915e228de62" (UID: "aa5086bc-fed4-481c-b531-7915e228de62"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:08.338490 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.338465 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa5086bc-fed4-481c-b531-7915e228de62-istio-token" (OuterVolumeSpecName: "istio-token") pod "aa5086bc-fed4-481c-b531-7915e228de62" (UID: "aa5086bc-fed4-481c-b531-7915e228de62"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:45:08.338577 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.338467 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa5086bc-fed4-481c-b531-7915e228de62-kube-api-access-8nhmh" (OuterVolumeSpecName: "kube-api-access-8nhmh") pod "aa5086bc-fed4-481c-b531-7915e228de62" (UID: "aa5086bc-fed4-481c-b531-7915e228de62"). InnerVolumeSpecName "kube-api-access-8nhmh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:45:08.338619 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.338599 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aa5086bc-fed4-481c-b531-7915e228de62" (UID: "aa5086bc-fed4-481c-b531-7915e228de62"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:08.338827 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.338802 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa5086bc-fed4-481c-b531-7915e228de62-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aa5086bc-fed4-481c-b531-7915e228de62" (UID: "aa5086bc-fed4-481c-b531-7915e228de62"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:45:08.438144 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.438105 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa5086bc-fed4-481c-b531-7915e228de62-istio-token\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:08.438144 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.438134 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nhmh\" (UniqueName: \"kubernetes.io/projected/aa5086bc-fed4-481c-b531-7915e228de62-kube-api-access-8nhmh\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:08.438144 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.438149 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa5086bc-fed4-481c-b531-7915e228de62-istio-podinfo\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:08.438388 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.438162 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-workload-socket\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:08.438388 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.438173 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-istio-data\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:08.438388 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.438184 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa5086bc-fed4-481c-b531-7915e228de62-istio-envoy\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:08.530069 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.530032 2581 generic.go:356] "Generic (PLEG): container finished" podID="aa5086bc-fed4-481c-b531-7915e228de62" containerID="bc13e616a9c85f2623c8e227c45d1250d94e5ac5908fc822fe3de02c94b54384" exitCode=0 Jul 09 14:45:08.530235 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.530086 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" Jul 09 14:45:08.530235 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.530114 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" event={"ID":"aa5086bc-fed4-481c-b531-7915e228de62","Type":"ContainerDied","Data":"bc13e616a9c85f2623c8e227c45d1250d94e5ac5908fc822fe3de02c94b54384"} Jul 09 14:45:08.530235 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.530172 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5" event={"ID":"aa5086bc-fed4-481c-b531-7915e228de62","Type":"ContainerDied","Data":"1e24ad4098e62fbfc7108c89f6a75bd42dc2d50707e9c67e3642421beb6929b2"} Jul 09 14:45:08.530235 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.530194 2581 scope.go:117] "RemoveContainer" containerID="bc13e616a9c85f2623c8e227c45d1250d94e5ac5908fc822fe3de02c94b54384" Jul 09 14:45:08.539788 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.539767 2581 scope.go:117] "RemoveContainer" containerID="8120816ab9b422f2cfc78f953828ef93eb981309b1829b42519b04e3ae0e5732" Jul 09 14:45:08.548033 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.548016 2581 scope.go:117] "RemoveContainer" containerID="bc13e616a9c85f2623c8e227c45d1250d94e5ac5908fc822fe3de02c94b54384" Jul 09 14:45:08.548319 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:45:08.548296 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc13e616a9c85f2623c8e227c45d1250d94e5ac5908fc822fe3de02c94b54384\": container with ID starting with bc13e616a9c85f2623c8e227c45d1250d94e5ac5908fc822fe3de02c94b54384 not found: ID does not exist" containerID="bc13e616a9c85f2623c8e227c45d1250d94e5ac5908fc822fe3de02c94b54384" Jul 09 14:45:08.548410 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.548335 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc13e616a9c85f2623c8e227c45d1250d94e5ac5908fc822fe3de02c94b54384"} err="failed to get container status \"bc13e616a9c85f2623c8e227c45d1250d94e5ac5908fc822fe3de02c94b54384\": rpc error: code = NotFound desc = could not find container \"bc13e616a9c85f2623c8e227c45d1250d94e5ac5908fc822fe3de02c94b54384\": container with ID starting with bc13e616a9c85f2623c8e227c45d1250d94e5ac5908fc822fe3de02c94b54384 not found: ID does not exist" Jul 09 14:45:08.548410 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.548362 2581 scope.go:117] "RemoveContainer" containerID="8120816ab9b422f2cfc78f953828ef93eb981309b1829b42519b04e3ae0e5732" Jul 09 14:45:08.548637 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:45:08.548619 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8120816ab9b422f2cfc78f953828ef93eb981309b1829b42519b04e3ae0e5732\": container with ID starting with 8120816ab9b422f2cfc78f953828ef93eb981309b1829b42519b04e3ae0e5732 not found: ID does not exist" containerID="8120816ab9b422f2cfc78f953828ef93eb981309b1829b42519b04e3ae0e5732" Jul 09 14:45:08.548692 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.548647 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8120816ab9b422f2cfc78f953828ef93eb981309b1829b42519b04e3ae0e5732"} err="failed to get container status \"8120816ab9b422f2cfc78f953828ef93eb981309b1829b42519b04e3ae0e5732\": rpc error: code = NotFound desc = could not find container \"8120816ab9b422f2cfc78f953828ef93eb981309b1829b42519b04e3ae0e5732\": container with ID starting with 8120816ab9b422f2cfc78f953828ef93eb981309b1829b42519b04e3ae0e5732 not found: ID does not exist" Jul 09 14:45:08.552316 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.552292 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5"] Jul 09 14:45:08.556614 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.556591 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-d2daa1-openshift-default-685c796df9-z86g5"] Jul 09 14:45:08.977778 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:08.977745 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa5086bc-fed4-481c-b531-7915e228de62" path="/var/lib/kubelet/pods/aa5086bc-fed4-481c-b531-7915e228de62/volumes" Jul 09 14:45:10.835245 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.835210 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq"] Jul 09 14:45:10.835650 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.835619 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5086bc-fed4-481c-b531-7915e228de62" containerName="istio-proxy" Jul 09 14:45:10.835650 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.835633 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5086bc-fed4-481c-b531-7915e228de62" containerName="istio-proxy" Jul 09 14:45:10.835650 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.835649 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c711ee7b-23f6-4c76-bf8f-4de23ec6b706" containerName="main" Jul 09 14:45:10.835753 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.835654 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c711ee7b-23f6-4c76-bf8f-4de23ec6b706" containerName="main" Jul 09 14:45:10.835753 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.835719 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa5086bc-fed4-481c-b531-7915e228de62" containerName="istio-proxy" Jul 09 14:45:10.835753 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.835729 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa5086bc-fed4-481c-b531-7915e228de62" containerName="istio-proxy" Jul 09 14:45:10.835753 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.835738 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c711ee7b-23f6-4c76-bf8f-4de23ec6b706" containerName="main" Jul 09 14:45:10.840555 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.840533 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:10.843395 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.843375 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-577c63-openshift-default-dockercfg-n5826\"" Jul 09 14:45:10.851884 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.851857 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq"] Jul 09 14:45:10.961471 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.961435 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-data\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:10.961649 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.961477 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-envoy\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:10.961649 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.961517 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-token\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:10.961649 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.961535 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/088845fe-3a3d-49a7-af26-4ebdaa595893-istiod-ca-cert\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:10.961649 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.961568 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-workload-certs\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:10.961649 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.961622 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-workload-socket\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:10.961649 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.961642 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-podinfo\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:10.961839 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.961718 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-credential-socket\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:10.961839 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:10.961769 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw7h2\" (UniqueName: \"kubernetes.io/projected/088845fe-3a3d-49a7-af26-4ebdaa595893-kube-api-access-xw7h2\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.062578 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.062540 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw7h2\" (UniqueName: \"kubernetes.io/projected/088845fe-3a3d-49a7-af26-4ebdaa595893-kube-api-access-xw7h2\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.062578 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.062585 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-data\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.062837 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.062703 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-envoy\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.062901 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.062854 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-token\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.062901 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.062880 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/088845fe-3a3d-49a7-af26-4ebdaa595893-istiod-ca-cert\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.063045 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.062907 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-data\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.063045 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.062944 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-workload-certs\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.063045 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.063021 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-workload-socket\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.063208 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.063053 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-podinfo\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.063208 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.063111 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-credential-socket\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.063208 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.063177 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-workload-certs\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.063433 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.063404 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-workload-socket\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.063526 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.063489 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-credential-socket\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.063661 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.063639 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/088845fe-3a3d-49a7-af26-4ebdaa595893-istiod-ca-cert\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.065086 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.065070 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-envoy\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.065481 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.065464 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-podinfo\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.071176 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.071155 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-token\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.071262 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.071230 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw7h2\" (UniqueName: \"kubernetes.io/projected/088845fe-3a3d-49a7-af26-4ebdaa595893-kube-api-access-xw7h2\") pod \"e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.151609 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.151581 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:11.296594 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.296559 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq"] Jul 09 14:45:11.298218 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:45:11.298182 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod088845fe_3a3d_49a7_af26_4ebdaa595893.slice/crio-94a8aa0feec07cd76dfc8c42672811cd6bb568998fa52466823240f041966f06 WatchSource:0}: Error finding container 94a8aa0feec07cd76dfc8c42672811cd6bb568998fa52466823240f041966f06: Status 404 returned error can't find the container with id 94a8aa0feec07cd76dfc8c42672811cd6bb568998fa52466823240f041966f06 Jul 09 14:45:11.300892 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.300854 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:11.300984 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.300920 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:11.300984 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.300972 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:11.549829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.549734 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" event={"ID":"088845fe-3a3d-49a7-af26-4ebdaa595893","Type":"ContainerStarted","Data":"dd4f954836b9bc5e9bffaf7d04e2dfe77581714edcf401317303b3f752d94d9a"} Jul 09 14:45:11.549829 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.549792 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" event={"ID":"088845fe-3a3d-49a7-af26-4ebdaa595893","Type":"ContainerStarted","Data":"94a8aa0feec07cd76dfc8c42672811cd6bb568998fa52466823240f041966f06"} Jul 09 14:45:11.571498 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:11.571445 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" podStartSLOduration=1.5714289350000001 podStartE2EDuration="1.571428935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:45:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:45:11.570162681 +0000 UTC m=+1831.693302079" watchObservedRunningTime="2026-07-09 14:45:11.571428935 +0000 UTC m=+1831.694568316" Jul 09 14:45:12.152136 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:12.152097 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:12.157193 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:12.157167 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:12.555286 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:12.555198 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:12.556400 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:12.556381 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:18.898400 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:18.898367 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb"] Jul 09 14:45:18.898850 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:18.898710 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5086bc-fed4-481c-b531-7915e228de62" containerName="istio-proxy" Jul 09 14:45:18.898850 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:18.898720 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5086bc-fed4-481c-b531-7915e228de62" containerName="istio-proxy" Jul 09 14:45:18.902240 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:18.902215 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:18.905194 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:18.905170 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-41b03c-openshift-default-dockercfg-jgpfc\"" Jul 09 14:45:18.926856 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:18.926828 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb"] Jul 09 14:45:19.046299 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.046259 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-istio-envoy\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.046469 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.046323 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-workload-certs\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.046469 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.046403 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-istio-data\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.046469 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.046461 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxkqz\" (UniqueName: \"kubernetes.io/projected/e88b1f44-7038-4397-9226-98bdd5f85131-kube-api-access-pxkqz\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.046580 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.046516 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e88b1f44-7038-4397-9226-98bdd5f85131-istiod-ca-cert\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.046615 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.046581 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e88b1f44-7038-4397-9226-98bdd5f85131-istio-token\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.046652 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.046627 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-workload-socket\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.046685 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.046653 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-credential-socket\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.046685 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.046674 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e88b1f44-7038-4397-9226-98bdd5f85131-istio-podinfo\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.147299 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.147268 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-istio-envoy\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.147486 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.147318 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-workload-certs\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.147486 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.147337 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-istio-data\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.147486 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.147459 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxkqz\" (UniqueName: \"kubernetes.io/projected/e88b1f44-7038-4397-9226-98bdd5f85131-kube-api-access-pxkqz\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.147671 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.147550 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e88b1f44-7038-4397-9226-98bdd5f85131-istiod-ca-cert\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.147671 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.147606 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e88b1f44-7038-4397-9226-98bdd5f85131-istio-token\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.147783 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.147679 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-workload-socket\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.147783 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.147707 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-credential-socket\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.147783 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.147734 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e88b1f44-7038-4397-9226-98bdd5f85131-istio-podinfo\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.147980 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.147816 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-workload-certs\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.148077 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.148001 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-istio-data\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.148223 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.148197 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-workload-socket\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.148290 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.148227 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-credential-socket\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.148483 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.148427 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e88b1f44-7038-4397-9226-98bdd5f85131-istiod-ca-cert\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.150360 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.150332 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-istio-envoy\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.150465 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.150410 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e88b1f44-7038-4397-9226-98bdd5f85131-istio-podinfo\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.156830 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.156806 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e88b1f44-7038-4397-9226-98bdd5f85131-istio-token\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.156930 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.156844 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxkqz\" (UniqueName: \"kubernetes.io/projected/e88b1f44-7038-4397-9226-98bdd5f85131-kube-api-access-pxkqz\") pod \"e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.213200 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.213172 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:19.366586 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.366560 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb"] Jul 09 14:45:19.367554 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:45:19.367524 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode88b1f44_7038_4397_9226_98bdd5f85131.slice/crio-204eba053fd5250552f85b3021d1dccaa2b2dcdc2208a4feadf4c0bb37424909 WatchSource:0}: Error finding container 204eba053fd5250552f85b3021d1dccaa2b2dcdc2208a4feadf4c0bb37424909: Status 404 returned error can't find the container with id 204eba053fd5250552f85b3021d1dccaa2b2dcdc2208a4feadf4c0bb37424909 Jul 09 14:45:19.369814 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.369780 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:19.369902 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.369849 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:19.369902 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.369880 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:19.592759 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.592719 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" event={"ID":"e88b1f44-7038-4397-9226-98bdd5f85131","Type":"ContainerStarted","Data":"4d0da0fe8ce41133190fd0e17950b25be281f0b13c10d0b19d1726443ffd003a"} Jul 09 14:45:19.592991 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.592777 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" event={"ID":"e88b1f44-7038-4397-9226-98bdd5f85131","Type":"ContainerStarted","Data":"204eba053fd5250552f85b3021d1dccaa2b2dcdc2208a4feadf4c0bb37424909"} Jul 09 14:45:19.614561 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:19.614506 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" podStartSLOduration=1.614489849 podStartE2EDuration="1.614489849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:45:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:45:19.612327867 +0000 UTC m=+1839.735467270" watchObservedRunningTime="2026-07-09 14:45:19.614489849 +0000 UTC m=+1839.737629232" Jul 09 14:45:20.213907 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:20.213875 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:20.219003 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:20.218977 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:20.598191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:20.598163 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:20.599043 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:20.599022 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:21.499209 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:21.499166 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:45:28.596126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:28.596082 2581 patch_prober.go:28] interesting pod/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 14:45:28.596520 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:28.596141 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" podUID="088845fe-3a3d-49a7-af26-4ebdaa595893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 14:45:29.647310 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:29.647278 2581 generic.go:356] "Generic (PLEG): container finished" podID="088845fe-3a3d-49a7-af26-4ebdaa595893" containerID="dd4f954836b9bc5e9bffaf7d04e2dfe77581714edcf401317303b3f752d94d9a" exitCode=0 Jul 09 14:45:29.647776 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:29.647359 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" event={"ID":"088845fe-3a3d-49a7-af26-4ebdaa595893","Type":"ContainerDied","Data":"dd4f954836b9bc5e9bffaf7d04e2dfe77581714edcf401317303b3f752d94d9a"} Jul 09 14:45:29.647840 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:29.647784 2581 scope.go:117] "RemoveContainer" containerID="dd4f954836b9bc5e9bffaf7d04e2dfe77581714edcf401317303b3f752d94d9a" Jul 09 14:45:29.648941 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:29.648911 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:29.649041 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:29.649024 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:29.649084 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:29.649077 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:30.654785 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:30.654749 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" event={"ID":"088845fe-3a3d-49a7-af26-4ebdaa595893","Type":"ContainerStarted","Data":"1308a3c6b6012fb71d7e9c9101548b19f10fe6ac26f3ce75d1b42d0d452db6cf"} Jul 09 14:45:31.152693 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:31.152650 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:31.152929 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:31.152902 2581 patch_prober.go:28] interesting pod/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 09 14:45:31.153044 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:31.152995 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" podUID="088845fe-3a3d-49a7-af26-4ebdaa595893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 09 14:45:32.152737 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:32.152695 2581 patch_prober.go:28] interesting pod/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 09 14:45:32.153238 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:32.152762 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" podUID="088845fe-3a3d-49a7-af26-4ebdaa595893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 09 14:45:33.152789 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:33.152750 2581 patch_prober.go:28] interesting pod/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 09 14:45:33.153314 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:33.152813 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" podUID="088845fe-3a3d-49a7-af26-4ebdaa595893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 09 14:45:34.152620 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:34.152575 2581 patch_prober.go:28] interesting pod/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 09 14:45:34.152824 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:34.152637 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" podUID="088845fe-3a3d-49a7-af26-4ebdaa595893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 09 14:45:35.155813 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:35.155773 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:35.156451 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:35.156418 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:35.157407 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:35.157386 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:36.640154 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:36.640105 2581 patch_prober.go:28] interesting pod/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 14:45:36.640548 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:36.640176 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" podUID="e88b1f44-7038-4397-9226-98bdd5f85131" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 14:45:38.155929 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:38.155897 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:45:38.706995 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:38.706937 2581 generic.go:356] "Generic (PLEG): container finished" podID="e88b1f44-7038-4397-9226-98bdd5f85131" containerID="4d0da0fe8ce41133190fd0e17950b25be281f0b13c10d0b19d1726443ffd003a" exitCode=0 Jul 09 14:45:38.707174 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:38.707016 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" event={"ID":"e88b1f44-7038-4397-9226-98bdd5f85131","Type":"ContainerDied","Data":"4d0da0fe8ce41133190fd0e17950b25be281f0b13c10d0b19d1726443ffd003a"} Jul 09 14:45:38.707438 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:38.707422 2581 scope.go:117] "RemoveContainer" containerID="4d0da0fe8ce41133190fd0e17950b25be281f0b13c10d0b19d1726443ffd003a" Jul 09 14:45:38.708420 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:38.708383 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:38.708510 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:38.708470 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:38.708510 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:38.708502 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:39.714766 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:39.714733 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" event={"ID":"e88b1f44-7038-4397-9226-98bdd5f85131","Type":"ContainerStarted","Data":"43caf8fbbdd386e20a9e95ce19f59eff6d5d3c4154a4b0ab1a635aae2f6aa30a"} Jul 09 14:45:40.213966 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:40.213916 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:40.214212 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:40.214188 2581 patch_prober.go:28] interesting pod/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 09 14:45:40.214301 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:40.214249 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" podUID="e88b1f44-7038-4397-9226-98bdd5f85131" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 09 14:45:41.214399 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:41.214361 2581 patch_prober.go:28] interesting pod/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 09 14:45:41.214779 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:41.214416 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" podUID="e88b1f44-7038-4397-9226-98bdd5f85131" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 09 14:45:41.323206 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:41.323170 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq"] Jul 09 14:45:41.323465 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:41.323433 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" podUID="088845fe-3a3d-49a7-af26-4ebdaa595893" containerName="istio-proxy" containerID="cri-o://1308a3c6b6012fb71d7e9c9101548b19f10fe6ac26f3ce75d1b42d0d452db6cf" gracePeriod=30 Jul 09 14:45:42.214044 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:42.214009 2581 patch_prober.go:28] interesting pod/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 09 14:45:42.214228 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:42.214072 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" podUID="e88b1f44-7038-4397-9226-98bdd5f85131" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 09 14:45:43.213709 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:43.213668 2581 patch_prober.go:28] interesting pod/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 09 14:45:43.214131 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:43.213728 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" podUID="e88b1f44-7038-4397-9226-98bdd5f85131" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 09 14:45:44.217125 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:44.217092 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:44.217521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:44.217459 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:44.218235 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:44.218215 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:46.588862 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.588840 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:46.740362 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.740310 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-credential-socket\") pod \"088845fe-3a3d-49a7-af26-4ebdaa595893\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " Jul 09 14:45:46.740362 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.740374 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-envoy\") pod \"088845fe-3a3d-49a7-af26-4ebdaa595893\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " Jul 09 14:45:46.740625 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.740397 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-workload-socket\") pod \"088845fe-3a3d-49a7-af26-4ebdaa595893\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " Jul 09 14:45:46.740625 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.740433 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-data\") pod \"088845fe-3a3d-49a7-af26-4ebdaa595893\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " Jul 09 14:45:46.740625 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.740449 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-token\") pod \"088845fe-3a3d-49a7-af26-4ebdaa595893\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " Jul 09 14:45:46.740625 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.740498 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-podinfo\") pod \"088845fe-3a3d-49a7-af26-4ebdaa595893\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " Jul 09 14:45:46.740625 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.740560 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/088845fe-3a3d-49a7-af26-4ebdaa595893-istiod-ca-cert\") pod \"088845fe-3a3d-49a7-af26-4ebdaa595893\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " Jul 09 14:45:46.740872 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.740719 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "088845fe-3a3d-49a7-af26-4ebdaa595893" (UID: "088845fe-3a3d-49a7-af26-4ebdaa595893"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:46.740872 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.740719 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "088845fe-3a3d-49a7-af26-4ebdaa595893" (UID: "088845fe-3a3d-49a7-af26-4ebdaa595893"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:46.740872 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.740747 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw7h2\" (UniqueName: \"kubernetes.io/projected/088845fe-3a3d-49a7-af26-4ebdaa595893-kube-api-access-xw7h2\") pod \"088845fe-3a3d-49a7-af26-4ebdaa595893\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " Jul 09 14:45:46.740872 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.740799 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-workload-certs\") pod \"088845fe-3a3d-49a7-af26-4ebdaa595893\" (UID: \"088845fe-3a3d-49a7-af26-4ebdaa595893\") " Jul 09 14:45:46.741143 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.741042 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "088845fe-3a3d-49a7-af26-4ebdaa595893" (UID: "088845fe-3a3d-49a7-af26-4ebdaa595893"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:46.741143 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.741071 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/088845fe-3a3d-49a7-af26-4ebdaa595893-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "088845fe-3a3d-49a7-af26-4ebdaa595893" (UID: "088845fe-3a3d-49a7-af26-4ebdaa595893"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:45:46.741323 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.741302 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/088845fe-3a3d-49a7-af26-4ebdaa595893-istiod-ca-cert\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:46.741386 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.741330 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-workload-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:46.741386 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.741344 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-credential-socket\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:46.741386 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.741357 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-workload-socket\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:46.741544 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.741489 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-data" (OuterVolumeSpecName: "istio-data") pod "088845fe-3a3d-49a7-af26-4ebdaa595893" (UID: "088845fe-3a3d-49a7-af26-4ebdaa595893"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:46.742938 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.742912 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-token" (OuterVolumeSpecName: "istio-token") pod "088845fe-3a3d-49a7-af26-4ebdaa595893" (UID: "088845fe-3a3d-49a7-af26-4ebdaa595893"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:45:46.743068 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.742934 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/088845fe-3a3d-49a7-af26-4ebdaa595893-kube-api-access-xw7h2" (OuterVolumeSpecName: "kube-api-access-xw7h2") pod "088845fe-3a3d-49a7-af26-4ebdaa595893" (UID: "088845fe-3a3d-49a7-af26-4ebdaa595893"). InnerVolumeSpecName "kube-api-access-xw7h2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:45:46.743227 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.743209 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "088845fe-3a3d-49a7-af26-4ebdaa595893" (UID: "088845fe-3a3d-49a7-af26-4ebdaa595893"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:45:46.743281 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.743226 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "088845fe-3a3d-49a7-af26-4ebdaa595893" (UID: "088845fe-3a3d-49a7-af26-4ebdaa595893"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:46.754215 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.754182 2581 generic.go:356] "Generic (PLEG): container finished" podID="088845fe-3a3d-49a7-af26-4ebdaa595893" containerID="1308a3c6b6012fb71d7e9c9101548b19f10fe6ac26f3ce75d1b42d0d452db6cf" exitCode=0 Jul 09 14:45:46.754343 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.754228 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" Jul 09 14:45:46.754343 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.754265 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" event={"ID":"088845fe-3a3d-49a7-af26-4ebdaa595893","Type":"ContainerDied","Data":"1308a3c6b6012fb71d7e9c9101548b19f10fe6ac26f3ce75d1b42d0d452db6cf"} Jul 09 14:45:46.754425 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.754341 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq" event={"ID":"088845fe-3a3d-49a7-af26-4ebdaa595893","Type":"ContainerDied","Data":"94a8aa0feec07cd76dfc8c42672811cd6bb568998fa52466823240f041966f06"} Jul 09 14:45:46.754425 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.754367 2581 scope.go:117] "RemoveContainer" containerID="1308a3c6b6012fb71d7e9c9101548b19f10fe6ac26f3ce75d1b42d0d452db6cf" Jul 09 14:45:46.767022 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.767003 2581 scope.go:117] "RemoveContainer" containerID="dd4f954836b9bc5e9bffaf7d04e2dfe77581714edcf401317303b3f752d94d9a" Jul 09 14:45:46.775531 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.775503 2581 scope.go:117] "RemoveContainer" containerID="1308a3c6b6012fb71d7e9c9101548b19f10fe6ac26f3ce75d1b42d0d452db6cf" Jul 09 14:45:46.775862 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:45:46.775822 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1308a3c6b6012fb71d7e9c9101548b19f10fe6ac26f3ce75d1b42d0d452db6cf\": container with ID starting with 1308a3c6b6012fb71d7e9c9101548b19f10fe6ac26f3ce75d1b42d0d452db6cf not found: ID does not exist" containerID="1308a3c6b6012fb71d7e9c9101548b19f10fe6ac26f3ce75d1b42d0d452db6cf" Jul 09 14:45:46.775959 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.775865 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1308a3c6b6012fb71d7e9c9101548b19f10fe6ac26f3ce75d1b42d0d452db6cf"} err="failed to get container status \"1308a3c6b6012fb71d7e9c9101548b19f10fe6ac26f3ce75d1b42d0d452db6cf\": rpc error: code = NotFound desc = could not find container \"1308a3c6b6012fb71d7e9c9101548b19f10fe6ac26f3ce75d1b42d0d452db6cf\": container with ID starting with 1308a3c6b6012fb71d7e9c9101548b19f10fe6ac26f3ce75d1b42d0d452db6cf not found: ID does not exist" Jul 09 14:45:46.775959 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.775890 2581 scope.go:117] "RemoveContainer" containerID="dd4f954836b9bc5e9bffaf7d04e2dfe77581714edcf401317303b3f752d94d9a" Jul 09 14:45:46.776256 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:45:46.776233 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd4f954836b9bc5e9bffaf7d04e2dfe77581714edcf401317303b3f752d94d9a\": container with ID starting with dd4f954836b9bc5e9bffaf7d04e2dfe77581714edcf401317303b3f752d94d9a not found: ID does not exist" containerID="dd4f954836b9bc5e9bffaf7d04e2dfe77581714edcf401317303b3f752d94d9a" Jul 09 14:45:46.776338 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.776263 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd4f954836b9bc5e9bffaf7d04e2dfe77581714edcf401317303b3f752d94d9a"} err="failed to get container status \"dd4f954836b9bc5e9bffaf7d04e2dfe77581714edcf401317303b3f752d94d9a\": rpc error: code = NotFound desc = could not find container \"dd4f954836b9bc5e9bffaf7d04e2dfe77581714edcf401317303b3f752d94d9a\": container with ID starting with dd4f954836b9bc5e9bffaf7d04e2dfe77581714edcf401317303b3f752d94d9a not found: ID does not exist" Jul 09 14:45:46.780224 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.780146 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq"] Jul 09 14:45:46.783741 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.783719 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-577c63-openshift-default-64c464c45c-8mzsq"] Jul 09 14:45:46.842453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.842343 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-data\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:46.842453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.842388 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-token\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:46.842453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.842400 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-podinfo\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:46.842453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.842412 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw7h2\" (UniqueName: \"kubernetes.io/projected/088845fe-3a3d-49a7-af26-4ebdaa595893-kube-api-access-xw7h2\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:46.842453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.842427 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/088845fe-3a3d-49a7-af26-4ebdaa595893-istio-envoy\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:46.979179 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:46.979134 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="088845fe-3a3d-49a7-af26-4ebdaa595893" path="/var/lib/kubelet/pods/088845fe-3a3d-49a7-af26-4ebdaa595893/volumes" Jul 09 14:45:48.411505 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:48.411469 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb"] Jul 09 14:45:48.411992 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:48.411738 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" podUID="e88b1f44-7038-4397-9226-98bdd5f85131" containerName="istio-proxy" containerID="cri-o://43caf8fbbdd386e20a9e95ce19f59eff6d5d3c4154a4b0ab1a635aae2f6aa30a" gracePeriod=30 Jul 09 14:45:51.089148 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:51.089111 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:45:53.683831 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.683763 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:53.794237 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.794205 2581 generic.go:356] "Generic (PLEG): container finished" podID="e88b1f44-7038-4397-9226-98bdd5f85131" containerID="43caf8fbbdd386e20a9e95ce19f59eff6d5d3c4154a4b0ab1a635aae2f6aa30a" exitCode=0 Jul 09 14:45:53.794418 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.794252 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" Jul 09 14:45:53.794418 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.794281 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" event={"ID":"e88b1f44-7038-4397-9226-98bdd5f85131","Type":"ContainerDied","Data":"43caf8fbbdd386e20a9e95ce19f59eff6d5d3c4154a4b0ab1a635aae2f6aa30a"} Jul 09 14:45:53.794418 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.794324 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb" event={"ID":"e88b1f44-7038-4397-9226-98bdd5f85131","Type":"ContainerDied","Data":"204eba053fd5250552f85b3021d1dccaa2b2dcdc2208a4feadf4c0bb37424909"} Jul 09 14:45:53.794418 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.794351 2581 scope.go:117] "RemoveContainer" containerID="43caf8fbbdd386e20a9e95ce19f59eff6d5d3c4154a4b0ab1a635aae2f6aa30a" Jul 09 14:45:53.803192 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.803173 2581 scope.go:117] "RemoveContainer" containerID="4d0da0fe8ce41133190fd0e17950b25be281f0b13c10d0b19d1726443ffd003a" Jul 09 14:45:53.803825 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.803807 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e88b1f44-7038-4397-9226-98bdd5f85131-istio-token\") pod \"e88b1f44-7038-4397-9226-98bdd5f85131\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " Jul 09 14:45:53.803919 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.803860 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-credential-socket\") pod \"e88b1f44-7038-4397-9226-98bdd5f85131\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " Jul 09 14:45:53.803919 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.803889 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-istio-envoy\") pod \"e88b1f44-7038-4397-9226-98bdd5f85131\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " Jul 09 14:45:53.803919 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.803906 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-istio-data\") pod \"e88b1f44-7038-4397-9226-98bdd5f85131\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " Jul 09 14:45:53.804085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.803928 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e88b1f44-7038-4397-9226-98bdd5f85131-istio-podinfo\") pod \"e88b1f44-7038-4397-9226-98bdd5f85131\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " Jul 09 14:45:53.804085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.803989 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e88b1f44-7038-4397-9226-98bdd5f85131-istiod-ca-cert\") pod \"e88b1f44-7038-4397-9226-98bdd5f85131\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " Jul 09 14:45:53.804085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.804017 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-workload-certs\") pod \"e88b1f44-7038-4397-9226-98bdd5f85131\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " Jul 09 14:45:53.804085 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.804048 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-workload-socket\") pod \"e88b1f44-7038-4397-9226-98bdd5f85131\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " Jul 09 14:45:53.804250 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.804136 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e88b1f44-7038-4397-9226-98bdd5f85131" (UID: "e88b1f44-7038-4397-9226-98bdd5f85131"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:53.804250 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.804158 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxkqz\" (UniqueName: \"kubernetes.io/projected/e88b1f44-7038-4397-9226-98bdd5f85131-kube-api-access-pxkqz\") pod \"e88b1f44-7038-4397-9226-98bdd5f85131\" (UID: \"e88b1f44-7038-4397-9226-98bdd5f85131\") " Jul 09 14:45:53.804872 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.804490 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e88b1f44-7038-4397-9226-98bdd5f85131-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e88b1f44-7038-4397-9226-98bdd5f85131" (UID: "e88b1f44-7038-4397-9226-98bdd5f85131"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:45:53.804872 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.804779 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e88b1f44-7038-4397-9226-98bdd5f85131" (UID: "e88b1f44-7038-4397-9226-98bdd5f85131"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:53.804872 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.804500 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-credential-socket\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:53.805117 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.804872 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e88b1f44-7038-4397-9226-98bdd5f85131" (UID: "e88b1f44-7038-4397-9226-98bdd5f85131"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:53.805346 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.805314 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-istio-data" (OuterVolumeSpecName: "istio-data") pod "e88b1f44-7038-4397-9226-98bdd5f85131" (UID: "e88b1f44-7038-4397-9226-98bdd5f85131"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:53.806482 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.806453 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e88b1f44-7038-4397-9226-98bdd5f85131-istio-token" (OuterVolumeSpecName: "istio-token") pod "e88b1f44-7038-4397-9226-98bdd5f85131" (UID: "e88b1f44-7038-4397-9226-98bdd5f85131"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:45:53.806622 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.806601 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e88b1f44-7038-4397-9226-98bdd5f85131-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e88b1f44-7038-4397-9226-98bdd5f85131" (UID: "e88b1f44-7038-4397-9226-98bdd5f85131"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:45:53.806622 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.806609 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e88b1f44-7038-4397-9226-98bdd5f85131" (UID: "e88b1f44-7038-4397-9226-98bdd5f85131"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:45:53.806878 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.806859 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e88b1f44-7038-4397-9226-98bdd5f85131-kube-api-access-pxkqz" (OuterVolumeSpecName: "kube-api-access-pxkqz") pod "e88b1f44-7038-4397-9226-98bdd5f85131" (UID: "e88b1f44-7038-4397-9226-98bdd5f85131"). InnerVolumeSpecName "kube-api-access-pxkqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:45:53.824283 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.824263 2581 scope.go:117] "RemoveContainer" containerID="43caf8fbbdd386e20a9e95ce19f59eff6d5d3c4154a4b0ab1a635aae2f6aa30a" Jul 09 14:45:53.824583 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:45:53.824564 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43caf8fbbdd386e20a9e95ce19f59eff6d5d3c4154a4b0ab1a635aae2f6aa30a\": container with ID starting with 43caf8fbbdd386e20a9e95ce19f59eff6d5d3c4154a4b0ab1a635aae2f6aa30a not found: ID does not exist" containerID="43caf8fbbdd386e20a9e95ce19f59eff6d5d3c4154a4b0ab1a635aae2f6aa30a" Jul 09 14:45:53.824664 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.824597 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43caf8fbbdd386e20a9e95ce19f59eff6d5d3c4154a4b0ab1a635aae2f6aa30a"} err="failed to get container status \"43caf8fbbdd386e20a9e95ce19f59eff6d5d3c4154a4b0ab1a635aae2f6aa30a\": rpc error: code = NotFound desc = could not find container \"43caf8fbbdd386e20a9e95ce19f59eff6d5d3c4154a4b0ab1a635aae2f6aa30a\": container with ID starting with 43caf8fbbdd386e20a9e95ce19f59eff6d5d3c4154a4b0ab1a635aae2f6aa30a not found: ID does not exist" Jul 09 14:45:53.824664 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.824621 2581 scope.go:117] "RemoveContainer" containerID="4d0da0fe8ce41133190fd0e17950b25be281f0b13c10d0b19d1726443ffd003a" Jul 09 14:45:53.824881 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:45:53.824864 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d0da0fe8ce41133190fd0e17950b25be281f0b13c10d0b19d1726443ffd003a\": container with ID starting with 4d0da0fe8ce41133190fd0e17950b25be281f0b13c10d0b19d1726443ffd003a not found: ID does not exist" containerID="4d0da0fe8ce41133190fd0e17950b25be281f0b13c10d0b19d1726443ffd003a" Jul 09 14:45:53.824925 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.824887 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d0da0fe8ce41133190fd0e17950b25be281f0b13c10d0b19d1726443ffd003a"} err="failed to get container status \"4d0da0fe8ce41133190fd0e17950b25be281f0b13c10d0b19d1726443ffd003a\": rpc error: code = NotFound desc = could not find container \"4d0da0fe8ce41133190fd0e17950b25be281f0b13c10d0b19d1726443ffd003a\": container with ID starting with 4d0da0fe8ce41133190fd0e17950b25be281f0b13c10d0b19d1726443ffd003a not found: ID does not exist" Jul 09 14:45:53.906240 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.906191 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxkqz\" (UniqueName: \"kubernetes.io/projected/e88b1f44-7038-4397-9226-98bdd5f85131-kube-api-access-pxkqz\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:53.906240 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.906229 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e88b1f44-7038-4397-9226-98bdd5f85131-istio-token\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:53.906240 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.906242 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-istio-envoy\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:53.906490 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.906254 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-istio-data\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:53.906490 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.906266 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e88b1f44-7038-4397-9226-98bdd5f85131-istio-podinfo\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:53.906490 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.906276 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e88b1f44-7038-4397-9226-98bdd5f85131-istiod-ca-cert\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:53.906490 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.906286 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-workload-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:53.906490 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:53.906298 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88b1f44-7038-4397-9226-98bdd5f85131-workload-socket\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:45:54.117785 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:54.117742 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb"] Jul 09 14:45:54.122173 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:54.122139 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-41b03c-openshift-default-5655c6fb55-kt4gb"] Jul 09 14:45:54.978206 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:54.978174 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e88b1f44-7038-4397-9226-98bdd5f85131" path="/var/lib/kubelet/pods/e88b1f44-7038-4397-9226-98bdd5f85131/volumes" Jul 09 14:45:58.081713 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.081680 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx"] Jul 09 14:45:58.082191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.082067 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e88b1f44-7038-4397-9226-98bdd5f85131" containerName="istio-proxy" Jul 09 14:45:58.082191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.082078 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e88b1f44-7038-4397-9226-98bdd5f85131" containerName="istio-proxy" Jul 09 14:45:58.082191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.082098 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e88b1f44-7038-4397-9226-98bdd5f85131" containerName="istio-proxy" Jul 09 14:45:58.082191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.082103 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e88b1f44-7038-4397-9226-98bdd5f85131" containerName="istio-proxy" Jul 09 14:45:58.082191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.082113 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="088845fe-3a3d-49a7-af26-4ebdaa595893" containerName="istio-proxy" Jul 09 14:45:58.082191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.082118 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="088845fe-3a3d-49a7-af26-4ebdaa595893" containerName="istio-proxy" Jul 09 14:45:58.082191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.082128 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="088845fe-3a3d-49a7-af26-4ebdaa595893" containerName="istio-proxy" Jul 09 14:45:58.082191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.082133 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="088845fe-3a3d-49a7-af26-4ebdaa595893" containerName="istio-proxy" Jul 09 14:45:58.082191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.082180 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e88b1f44-7038-4397-9226-98bdd5f85131" containerName="istio-proxy" Jul 09 14:45:58.082191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.082188 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e88b1f44-7038-4397-9226-98bdd5f85131" containerName="istio-proxy" Jul 09 14:45:58.082191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.082194 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="088845fe-3a3d-49a7-af26-4ebdaa595893" containerName="istio-proxy" Jul 09 14:45:58.082627 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.082203 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="088845fe-3a3d-49a7-af26-4ebdaa595893" containerName="istio-proxy" Jul 09 14:45:58.086677 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.086653 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.089487 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.089463 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-c4eda7-openshift-default-dockercfg-dzfmp\"" Jul 09 14:45:58.098455 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.098426 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx"] Jul 09 14:45:58.140541 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.140501 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-workload-socket\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.140709 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.140563 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/586763e8-1619-4c6f-843b-f93b6b02bb4f-istiod-ca-cert\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.140709 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.140609 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-772zg\" (UniqueName: \"kubernetes.io/projected/586763e8-1619-4c6f-843b-f93b6b02bb4f-kube-api-access-772zg\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.140709 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.140641 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-token\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.140709 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.140689 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-podinfo\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.140844 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.140732 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-envoy\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.140844 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.140833 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-data\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.140906 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.140857 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-workload-certs\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.140906 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.140890 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-credential-socket\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.241525 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.241489 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-podinfo\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.241719 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.241536 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-envoy\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.241719 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.241623 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-data\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.241719 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.241647 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-workload-certs\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.241894 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.241746 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-credential-socket\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.241894 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.241817 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-workload-socket\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.241894 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.241874 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/586763e8-1619-4c6f-843b-f93b6b02bb4f-istiod-ca-cert\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.242075 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.241905 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-772zg\" (UniqueName: \"kubernetes.io/projected/586763e8-1619-4c6f-843b-f93b6b02bb4f-kube-api-access-772zg\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.242075 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.241945 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-token\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.242195 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.242158 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-workload-certs\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.242195 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.242179 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-data\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.242446 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.242421 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-workload-socket\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.242539 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.242464 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-credential-socket\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.243035 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.243010 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/586763e8-1619-4c6f-843b-f93b6b02bb4f-istiod-ca-cert\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.244521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.244498 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-envoy\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.244740 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.244721 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-podinfo\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.251308 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.251282 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-token\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.251495 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.251476 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-772zg\" (UniqueName: \"kubernetes.io/projected/586763e8-1619-4c6f-843b-f93b6b02bb4f-kube-api-access-772zg\") pod \"e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.398158 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.398125 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:58.539031 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.538997 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx"] Jul 09 14:45:58.540274 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:45:58.540244 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod586763e8_1619_4c6f_843b_f93b6b02bb4f.slice/crio-4154254d97507df0761ad6b96f0778e5a1ef358091ca97acade2b265406769b2 WatchSource:0}: Error finding container 4154254d97507df0761ad6b96f0778e5a1ef358091ca97acade2b265406769b2: Status 404 returned error can't find the container with id 4154254d97507df0761ad6b96f0778e5a1ef358091ca97acade2b265406769b2 Jul 09 14:45:58.542662 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.542622 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:58.542728 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.542711 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:58.542762 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.542751 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:45:58.825405 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.825311 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" event={"ID":"586763e8-1619-4c6f-843b-f93b6b02bb4f","Type":"ContainerStarted","Data":"858231b5bbd658b1724ef9faf3f5e0648a648e192e93b159785bf02f2d9df91f"} Jul 09 14:45:58.825405 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.825367 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" event={"ID":"586763e8-1619-4c6f-843b-f93b6b02bb4f","Type":"ContainerStarted","Data":"4154254d97507df0761ad6b96f0778e5a1ef358091ca97acade2b265406769b2"} Jul 09 14:45:58.848003 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:58.847941 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" podStartSLOduration=0.847927779 podStartE2EDuration="847.927779ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:45:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:45:58.845137907 +0000 UTC m=+1878.968277290" watchObservedRunningTime="2026-07-09 14:45:58.847927779 +0000 UTC m=+1878.971067162" Jul 09 14:45:59.398897 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:59.398858 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:59.404021 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:59.403994 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:59.830795 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:59.830700 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:45:59.831683 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:45:59.831663 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:46:04.392572 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:04.392478 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:46:12.914646 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:12.914611 2581 generic.go:356] "Generic (PLEG): container finished" podID="586763e8-1619-4c6f-843b-f93b6b02bb4f" containerID="858231b5bbd658b1724ef9faf3f5e0648a648e192e93b159785bf02f2d9df91f" exitCode=0 Jul 09 14:46:12.914646 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:12.914655 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" event={"ID":"586763e8-1619-4c6f-843b-f93b6b02bb4f","Type":"ContainerDied","Data":"858231b5bbd658b1724ef9faf3f5e0648a648e192e93b159785bf02f2d9df91f"} Jul 09 14:46:12.915272 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:12.915186 2581 scope.go:117] "RemoveContainer" containerID="858231b5bbd658b1724ef9faf3f5e0648a648e192e93b159785bf02f2d9df91f" Jul 09 14:46:12.916323 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:12.916292 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:46:12.916409 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:12.916382 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:46:12.916464 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:12.916447 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 09 14:46:13.921864 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:13.921821 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" event={"ID":"586763e8-1619-4c6f-843b-f93b6b02bb4f","Type":"ContainerStarted","Data":"8b16b4b9e6b2ca431bd93bb263c3fb8b8a6a1d9530867df018b0623e45495328"} Jul 09 14:46:14.399133 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:14.399088 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:46:14.400281 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:14.400255 2581 patch_prober.go:28] interesting pod/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 09 14:46:14.400339 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:14.400310 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" podUID="586763e8-1619-4c6f-843b-f93b6b02bb4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 09 14:46:14.830741 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:14.830652 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:46:15.398871 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:15.398826 2581 patch_prober.go:28] interesting pod/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 09 14:46:15.399296 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:15.398887 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" podUID="586763e8-1619-4c6f-843b-f93b6b02bb4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 09 14:46:16.398929 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:16.398879 2581 patch_prober.go:28] interesting pod/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 09 14:46:16.399359 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:16.398985 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" podUID="586763e8-1619-4c6f-843b-f93b6b02bb4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 09 14:46:17.399247 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:17.399208 2581 patch_prober.go:28] interesting pod/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 09 14:46:17.399655 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:17.399278 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" podUID="586763e8-1619-4c6f-843b-f93b6b02bb4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 09 14:46:18.403701 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:18.403669 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:46:18.404711 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:18.404689 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:46:19.175498 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.175464 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs"] Jul 09 14:46:19.178045 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.178023 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.183545 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.183519 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b22ad3\"/\"default-dockercfg-ltx96\"" Jul 09 14:46:19.183704 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.183552 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b22ad3\"/\"auth-test-model-b22ad3-kserve-self-signed-certs\"" Jul 09 14:46:19.183704 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.183556 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b22ad3\"/\"kube-root-ca.crt\"" Jul 09 14:46:19.183704 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.183554 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b22ad3\"/\"openshift-service-ca.crt\"" Jul 09 14:46:19.191236 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.191211 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs"] Jul 09 14:46:19.251814 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.251772 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-home\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.252029 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.251827 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-dshm\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.252029 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.251912 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-tmp-dir\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.252029 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.252016 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt66b\" (UniqueName: \"kubernetes.io/projected/bfad1e41-e6ae-4693-b950-d89c2da0d42e-kube-api-access-mt66b\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.252157 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.252050 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-model-cache\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.252157 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.252103 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfad1e41-e6ae-4693-b950-d89c2da0d42e-tls-certs\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.352726 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.352685 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt66b\" (UniqueName: \"kubernetes.io/projected/bfad1e41-e6ae-4693-b950-d89c2da0d42e-kube-api-access-mt66b\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.352726 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.352729 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-model-cache\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.353043 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.352764 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfad1e41-e6ae-4693-b950-d89c2da0d42e-tls-certs\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.353043 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.352824 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-home\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.353043 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.352868 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-dshm\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.353043 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.352921 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-tmp-dir\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.353281 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.353258 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-model-cache\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.353340 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.353295 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-home\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.353403 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.353368 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-tmp-dir\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.355191 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.355172 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-dshm\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.355449 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.355431 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfad1e41-e6ae-4693-b950-d89c2da0d42e-tls-certs\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.364813 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.364781 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt66b\" (UniqueName: \"kubernetes.io/projected/bfad1e41-e6ae-4693-b950-d89c2da0d42e-kube-api-access-mt66b\") pod \"auth-test-model-b22ad3-kserve-85945d79f-8dmgs\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.488370 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.488274 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:19.638907 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.638877 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs"] Jul 09 14:46:19.640335 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:46:19.640304 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfad1e41_e6ae_4693_b950_d89c2da0d42e.slice/crio-ca6001e71fab68ccebf7d56a7dabdbe54c93e39244bf527f04dda66bda8d4648 WatchSource:0}: Error finding container ca6001e71fab68ccebf7d56a7dabdbe54c93e39244bf527f04dda66bda8d4648: Status 404 returned error can't find the container with id ca6001e71fab68ccebf7d56a7dabdbe54c93e39244bf527f04dda66bda8d4648 Jul 09 14:46:19.955972 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.955902 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" event={"ID":"bfad1e41-e6ae-4693-b950-d89c2da0d42e","Type":"ContainerStarted","Data":"c8990be0cd229be9c67ca4d11815e73fe4b7c9dc8a1d0b00be81f4a2ebcdd1cc"} Jul 09 14:46:19.955972 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.955968 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" event={"ID":"bfad1e41-e6ae-4693-b950-d89c2da0d42e","Type":"ContainerStarted","Data":"ca6001e71fab68ccebf7d56a7dabdbe54c93e39244bf527f04dda66bda8d4648"} Jul 09 14:46:19.982585 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:19.982532 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" podStartSLOduration=0.98251745 podStartE2EDuration="982.51745ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:46:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:46:19.982454378 +0000 UTC m=+1900.105593760" watchObservedRunningTime="2026-07-09 14:46:19.98251745 +0000 UTC m=+1900.105656833" Jul 09 14:46:22.812066 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:22.812029 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:46:29.488727 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:29.488683 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:29.489177 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:29.488741 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:29.501367 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:29.501342 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:30.022868 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:30.022833 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:46:41.505928 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:41.505886 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:46:59.516968 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:46:59.516929 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:47:15.097608 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:47:15.097571 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:47:32.330103 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:47:32.330024 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:47:49.906114 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:47:49.906075 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:48:06.901495 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:06.901457 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:48:22.294934 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:22.294897 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:48:39.905840 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:39.905801 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:48:44.723339 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:44.723304 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs"] Jul 09 14:48:50.284893 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.284857 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w"] Jul 09 14:48:50.287504 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.287480 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.290515 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.290494 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b22ad3\"/\"sub-test-model-b22ad3-kserve-self-signed-certs\"" Jul 09 14:48:50.298789 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.298766 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w"] Jul 09 14:48:50.415246 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.415209 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-dshm\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.415423 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.415264 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-home\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.415423 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.415294 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-model-cache\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.415423 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.415320 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12a1858a-b441-4b77-9ec2-a801fe5666a0-tls-certs\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.415423 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.415358 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hjhm\" (UniqueName: \"kubernetes.io/projected/12a1858a-b441-4b77-9ec2-a801fe5666a0-kube-api-access-6hjhm\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.415423 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.415395 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-tmp-dir\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.516498 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.516457 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-home\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.516498 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.516500 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-model-cache\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.516741 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.516531 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12a1858a-b441-4b77-9ec2-a801fe5666a0-tls-certs\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.516741 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.516585 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hjhm\" (UniqueName: \"kubernetes.io/projected/12a1858a-b441-4b77-9ec2-a801fe5666a0-kube-api-access-6hjhm\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.516741 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.516646 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-tmp-dir\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.516741 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.516693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-dshm\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.516979 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.516905 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-home\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.517052 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.516981 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-model-cache\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.517110 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.517065 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-tmp-dir\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.518864 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.518845 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-dshm\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.519073 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.519054 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12a1858a-b441-4b77-9ec2-a801fe5666a0-tls-certs\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.527593 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.527570 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hjhm\" (UniqueName: \"kubernetes.io/projected/12a1858a-b441-4b77-9ec2-a801fe5666a0-kube-api-access-6hjhm\") pod \"sub-test-model-b22ad3-kserve-7964978b5f-wl54w\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.597603 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.597568 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:48:50.735358 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.735325 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w"] Jul 09 14:48:50.737312 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:48:50.737277 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12a1858a_b441_4b77_9ec2_a801fe5666a0.slice/crio-0abe1585ab94761bc5100c70b4f12f86e4bdd3235a1bdcb576375c609238c253 WatchSource:0}: Error finding container 0abe1585ab94761bc5100c70b4f12f86e4bdd3235a1bdcb576375c609238c253: Status 404 returned error can't find the container with id 0abe1585ab94761bc5100c70b4f12f86e4bdd3235a1bdcb576375c609238c253 Jul 09 14:48:50.788550 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:50.788523 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" event={"ID":"12a1858a-b441-4b77-9ec2-a801fe5666a0","Type":"ContainerStarted","Data":"0abe1585ab94761bc5100c70b4f12f86e4bdd3235a1bdcb576375c609238c253"} Jul 09 14:48:51.795411 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:51.795370 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" event={"ID":"12a1858a-b441-4b77-9ec2-a801fe5666a0","Type":"ContainerStarted","Data":"3eecb56be133707ad147e7457f7892f2c0ef2b6896fb4535e9a84eb70294518e"} Jul 09 14:48:51.815569 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:51.815510 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" podStartSLOduration=1.815492378 podStartE2EDuration="1.815492378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:48:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:48:51.813613365 +0000 UTC m=+2051.936752772" watchObservedRunningTime="2026-07-09 14:48:51.815492378 +0000 UTC m=+2051.938631772" Jul 09 14:48:59.298267 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.298229 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:48:59.743453 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.743402 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" podUID="bfad1e41-e6ae-4693-b950-d89c2da0d42e" containerName="main" containerID="cri-o://c8990be0cd229be9c67ca4d11815e73fe4b7c9dc8a1d0b00be81f4a2ebcdd1cc" gracePeriod=45 Jul 09 14:48:59.997255 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.997190 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:48:59.998731 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.998707 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-tmp-dir\") pod \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " Jul 09 14:48:59.998871 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.998758 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfad1e41-e6ae-4693-b950-d89c2da0d42e-tls-certs\") pod \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " Jul 09 14:48:59.998871 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.998779 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-home\") pod \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " Jul 09 14:48:59.998871 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.998816 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-dshm\") pod \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " Jul 09 14:48:59.999072 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.998981 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-model-cache\") pod \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " Jul 09 14:48:59.999072 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.998999 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bfad1e41-e6ae-4693-b950-d89c2da0d42e" (UID: "bfad1e41-e6ae-4693-b950-d89c2da0d42e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:48:59.999072 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.999035 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-home" (OuterVolumeSpecName: "home") pod "bfad1e41-e6ae-4693-b950-d89c2da0d42e" (UID: "bfad1e41-e6ae-4693-b950-d89c2da0d42e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:48:59.999072 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.999055 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt66b\" (UniqueName: \"kubernetes.io/projected/bfad1e41-e6ae-4693-b950-d89c2da0d42e-kube-api-access-mt66b\") pod \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\" (UID: \"bfad1e41-e6ae-4693-b950-d89c2da0d42e\") " Jul 09 14:48:59.999274 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.999224 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-model-cache" (OuterVolumeSpecName: "model-cache") pod "bfad1e41-e6ae-4693-b950-d89c2da0d42e" (UID: "bfad1e41-e6ae-4693-b950-d89c2da0d42e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:48:59.999498 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.999475 2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:48:59.999498 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.999499 2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:48:59.999689 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:48:59.999513 2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:49:00.001238 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.001204 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfad1e41-e6ae-4693-b950-d89c2da0d42e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bfad1e41-e6ae-4693-b950-d89c2da0d42e" (UID: "bfad1e41-e6ae-4693-b950-d89c2da0d42e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:49:00.001355 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.001274 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-dshm" (OuterVolumeSpecName: "dshm") pod "bfad1e41-e6ae-4693-b950-d89c2da0d42e" (UID: "bfad1e41-e6ae-4693-b950-d89c2da0d42e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:49:00.001355 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.001287 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfad1e41-e6ae-4693-b950-d89c2da0d42e-kube-api-access-mt66b" (OuterVolumeSpecName: "kube-api-access-mt66b") pod "bfad1e41-e6ae-4693-b950-d89c2da0d42e" (UID: "bfad1e41-e6ae-4693-b950-d89c2da0d42e"). InnerVolumeSpecName "kube-api-access-mt66b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:49:00.100493 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.100454 2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfad1e41-e6ae-4693-b950-d89c2da0d42e-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:49:00.100493 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.100485 2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfad1e41-e6ae-4693-b950-d89c2da0d42e-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:49:00.100493 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.100494 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt66b\" (UniqueName: \"kubernetes.io/projected/bfad1e41-e6ae-4693-b950-d89c2da0d42e-kube-api-access-mt66b\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:49:00.598710 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.598667 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:49:00.598710 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.598715 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:49:00.611238 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.611210 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:49:00.850262 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.850175 2581 generic.go:356] "Generic (PLEG): container finished" podID="bfad1e41-e6ae-4693-b950-d89c2da0d42e" containerID="c8990be0cd229be9c67ca4d11815e73fe4b7c9dc8a1d0b00be81f4a2ebcdd1cc" exitCode=0 Jul 09 14:49:00.850262 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.850237 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" Jul 09 14:49:00.850487 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.850264 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" event={"ID":"bfad1e41-e6ae-4693-b950-d89c2da0d42e","Type":"ContainerDied","Data":"c8990be0cd229be9c67ca4d11815e73fe4b7c9dc8a1d0b00be81f4a2ebcdd1cc"} Jul 09 14:49:00.850487 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.850318 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs" event={"ID":"bfad1e41-e6ae-4693-b950-d89c2da0d42e","Type":"ContainerDied","Data":"ca6001e71fab68ccebf7d56a7dabdbe54c93e39244bf527f04dda66bda8d4648"} Jul 09 14:49:00.850487 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.850344 2581 scope.go:117] "RemoveContainer" containerID="c8990be0cd229be9c67ca4d11815e73fe4b7c9dc8a1d0b00be81f4a2ebcdd1cc" Jul 09 14:49:00.861359 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.861337 2581 scope.go:117] "RemoveContainer" containerID="c8990be0cd229be9c67ca4d11815e73fe4b7c9dc8a1d0b00be81f4a2ebcdd1cc" Jul 09 14:49:00.861702 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:49:00.861669 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8990be0cd229be9c67ca4d11815e73fe4b7c9dc8a1d0b00be81f4a2ebcdd1cc\": container with ID starting with c8990be0cd229be9c67ca4d11815e73fe4b7c9dc8a1d0b00be81f4a2ebcdd1cc not found: ID does not exist" containerID="c8990be0cd229be9c67ca4d11815e73fe4b7c9dc8a1d0b00be81f4a2ebcdd1cc" Jul 09 14:49:00.861818 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.861714 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8990be0cd229be9c67ca4d11815e73fe4b7c9dc8a1d0b00be81f4a2ebcdd1cc"} err="failed to get container status \"c8990be0cd229be9c67ca4d11815e73fe4b7c9dc8a1d0b00be81f4a2ebcdd1cc\": rpc error: code = NotFound desc = could not find container \"c8990be0cd229be9c67ca4d11815e73fe4b7c9dc8a1d0b00be81f4a2ebcdd1cc\": container with ID starting with c8990be0cd229be9c67ca4d11815e73fe4b7c9dc8a1d0b00be81f4a2ebcdd1cc not found: ID does not exist" Jul 09 14:49:00.863447 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.863421 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:49:00.874510 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.874478 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs"] Jul 09 14:49:00.878454 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.878431 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b22ad3/auth-test-model-b22ad3-kserve-85945d79f-8dmgs"] Jul 09 14:49:00.977591 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:00.977554 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfad1e41-e6ae-4693-b950-d89c2da0d42e" path="/var/lib/kubelet/pods/bfad1e41-e6ae-4693-b950-d89c2da0d42e/volumes" Jul 09 14:49:31.600126 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:31.600088 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:49:40.845643 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:40.845611 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:49:40.852309 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:40.852284 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v67ml_769bd7fc-a76c-4797-b56d-30577140294f/ovn-acl-logging/0.log" Jul 09 14:49:56.509314 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:49:56.509276 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:50:15.010086 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:15.010048 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:50:20.353197 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:20.353160 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w"] Jul 09 14:50:34.608804 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:34.608719 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:50:35.372974 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.372906 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" podUID="12a1858a-b441-4b77-9ec2-a801fe5666a0" containerName="main" containerID="cri-o://3eecb56be133707ad147e7457f7892f2c0ef2b6896fb4535e9a84eb70294518e" gracePeriod=45 Jul 09 14:50:35.639881 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.639854 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:50:35.735630 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.735600 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-model-cache\") pod \"12a1858a-b441-4b77-9ec2-a801fe5666a0\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " Jul 09 14:50:35.735830 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.735658 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-tmp-dir\") pod \"12a1858a-b441-4b77-9ec2-a801fe5666a0\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " Jul 09 14:50:35.735830 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.735692 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hjhm\" (UniqueName: \"kubernetes.io/projected/12a1858a-b441-4b77-9ec2-a801fe5666a0-kube-api-access-6hjhm\") pod \"12a1858a-b441-4b77-9ec2-a801fe5666a0\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " Jul 09 14:50:35.735830 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.735720 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12a1858a-b441-4b77-9ec2-a801fe5666a0-tls-certs\") pod \"12a1858a-b441-4b77-9ec2-a801fe5666a0\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " Jul 09 14:50:35.735830 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.735780 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-dshm\") pod \"12a1858a-b441-4b77-9ec2-a801fe5666a0\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " Jul 09 14:50:35.735830 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.735821 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-home\") pod \"12a1858a-b441-4b77-9ec2-a801fe5666a0\" (UID: \"12a1858a-b441-4b77-9ec2-a801fe5666a0\") " Jul 09 14:50:35.736142 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.735900 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-model-cache" (OuterVolumeSpecName: "model-cache") pod "12a1858a-b441-4b77-9ec2-a801fe5666a0" (UID: "12a1858a-b441-4b77-9ec2-a801fe5666a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:50:35.736142 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.735994 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "12a1858a-b441-4b77-9ec2-a801fe5666a0" (UID: "12a1858a-b441-4b77-9ec2-a801fe5666a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:50:35.736142 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.736108 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-home" (OuterVolumeSpecName: "home") pod "12a1858a-b441-4b77-9ec2-a801fe5666a0" (UID: "12a1858a-b441-4b77-9ec2-a801fe5666a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:50:35.736292 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.736222 2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:50:35.736292 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.736240 2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:50:35.736292 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.736255 2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:50:35.738007 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.737980 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12a1858a-b441-4b77-9ec2-a801fe5666a0-kube-api-access-6hjhm" (OuterVolumeSpecName: "kube-api-access-6hjhm") pod "12a1858a-b441-4b77-9ec2-a801fe5666a0" (UID: "12a1858a-b441-4b77-9ec2-a801fe5666a0"). InnerVolumeSpecName "kube-api-access-6hjhm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:50:35.738007 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.737985 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12a1858a-b441-4b77-9ec2-a801fe5666a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "12a1858a-b441-4b77-9ec2-a801fe5666a0" (UID: "12a1858a-b441-4b77-9ec2-a801fe5666a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:50:35.738172 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.738061 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-dshm" (OuterVolumeSpecName: "dshm") pod "12a1858a-b441-4b77-9ec2-a801fe5666a0" (UID: "12a1858a-b441-4b77-9ec2-a801fe5666a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:50:35.837078 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.837038 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hjhm\" (UniqueName: \"kubernetes.io/projected/12a1858a-b441-4b77-9ec2-a801fe5666a0-kube-api-access-6hjhm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:50:35.837078 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.837070 2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12a1858a-b441-4b77-9ec2-a801fe5666a0-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:50:35.837078 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:35.837080 2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12a1858a-b441-4b77-9ec2-a801fe5666a0-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:50:36.368200 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:36.368161 2581 generic.go:356] "Generic (PLEG): container finished" podID="12a1858a-b441-4b77-9ec2-a801fe5666a0" containerID="3eecb56be133707ad147e7457f7892f2c0ef2b6896fb4535e9a84eb70294518e" exitCode=0 Jul 09 14:50:36.368200 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:36.368199 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" Jul 09 14:50:36.368433 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:36.368249 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" event={"ID":"12a1858a-b441-4b77-9ec2-a801fe5666a0","Type":"ContainerDied","Data":"3eecb56be133707ad147e7457f7892f2c0ef2b6896fb4535e9a84eb70294518e"} Jul 09 14:50:36.368433 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:36.368293 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w" event={"ID":"12a1858a-b441-4b77-9ec2-a801fe5666a0","Type":"ContainerDied","Data":"0abe1585ab94761bc5100c70b4f12f86e4bdd3235a1bdcb576375c609238c253"} Jul 09 14:50:36.368433 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:36.368315 2581 scope.go:117] "RemoveContainer" containerID="3eecb56be133707ad147e7457f7892f2c0ef2b6896fb4535e9a84eb70294518e" Jul 09 14:50:36.378253 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:36.378097 2581 scope.go:117] "RemoveContainer" containerID="3eecb56be133707ad147e7457f7892f2c0ef2b6896fb4535e9a84eb70294518e" Jul 09 14:50:36.378413 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:50:36.378392 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3eecb56be133707ad147e7457f7892f2c0ef2b6896fb4535e9a84eb70294518e\": container with ID starting with 3eecb56be133707ad147e7457f7892f2c0ef2b6896fb4535e9a84eb70294518e not found: ID does not exist" containerID="3eecb56be133707ad147e7457f7892f2c0ef2b6896fb4535e9a84eb70294518e" Jul 09 14:50:36.378474 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:36.378428 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3eecb56be133707ad147e7457f7892f2c0ef2b6896fb4535e9a84eb70294518e"} err="failed to get container status \"3eecb56be133707ad147e7457f7892f2c0ef2b6896fb4535e9a84eb70294518e\": rpc error: code = NotFound desc = could not find container \"3eecb56be133707ad147e7457f7892f2c0ef2b6896fb4535e9a84eb70294518e\": container with ID starting with 3eecb56be133707ad147e7457f7892f2c0ef2b6896fb4535e9a84eb70294518e not found: ID does not exist" Jul 09 14:50:36.390676 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:36.390649 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w"] Jul 09 14:50:36.401925 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:36.401900 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b22ad3/sub-test-model-b22ad3-kserve-7964978b5f-wl54w"] Jul 09 14:50:36.978139 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:50:36.978105 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12a1858a-b441-4b77-9ec2-a801fe5666a0" path="/var/lib/kubelet/pods/12a1858a-b441-4b77-9ec2-a801fe5666a0/volumes" Jul 09 14:51:02.394657 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.394616 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx"] Jul 09 14:51:02.395143 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.394999 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfad1e41-e6ae-4693-b950-d89c2da0d42e" containerName="main" Jul 09 14:51:02.395143 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.395012 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfad1e41-e6ae-4693-b950-d89c2da0d42e" containerName="main" Jul 09 14:51:02.395143 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.395022 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12a1858a-b441-4b77-9ec2-a801fe5666a0" containerName="main" Jul 09 14:51:02.395143 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.395029 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="12a1858a-b441-4b77-9ec2-a801fe5666a0" containerName="main" Jul 09 14:51:02.395143 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.395096 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfad1e41-e6ae-4693-b950-d89c2da0d42e" containerName="main" Jul 09 14:51:02.395143 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.395105 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="12a1858a-b441-4b77-9ec2-a801fe5666a0" containerName="main" Jul 09 14:51:02.398213 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.398195 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.403249 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.403221 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c4eda7\"/\"kube-root-ca.crt\"" Jul 09 14:51:02.403381 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.403261 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c4eda7\"/\"openshift-service-ca.crt\"" Jul 09 14:51:02.403381 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.403361 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c4eda7\"/\"rate-test-model-c4eda7-kserve-self-signed-certs\"" Jul 09 14:51:02.404059 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.404039 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c4eda7\"/\"default-dockercfg-kqnqp\"" Jul 09 14:51:02.419937 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.419910 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx"] Jul 09 14:51:02.505059 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.505028 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-model-cache\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.505251 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.505154 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-tmp-dir\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.505251 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.505192 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b56dadd-0bce-4d6c-8041-f8dc3798e075-tls-certs\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.505251 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.505226 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm2r9\" (UniqueName: \"kubernetes.io/projected/4b56dadd-0bce-4d6c-8041-f8dc3798e075-kube-api-access-nm2r9\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.505421 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.505264 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-dshm\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.505421 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.505326 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-home\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.606801 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.606762 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-tmp-dir\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.607023 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.606809 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b56dadd-0bce-4d6c-8041-f8dc3798e075-tls-certs\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.607023 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.606909 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm2r9\" (UniqueName: \"kubernetes.io/projected/4b56dadd-0bce-4d6c-8041-f8dc3798e075-kube-api-access-nm2r9\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.607023 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.606990 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-dshm\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.607211 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.607045 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-home\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.607211 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.607112 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-model-cache\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.607307 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.607247 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-tmp-dir\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.607555 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.607531 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-model-cache\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.607656 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.607604 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-home\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.609392 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.609364 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-dshm\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.609683 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.609659 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b56dadd-0bce-4d6c-8041-f8dc3798e075-tls-certs\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.617778 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.617750 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm2r9\" (UniqueName: \"kubernetes.io/projected/4b56dadd-0bce-4d6c-8041-f8dc3798e075-kube-api-access-nm2r9\") pod \"rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.707379 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.707287 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:02.854227 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:02.854197 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx"] Jul 09 14:51:02.856510 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:51:02.856483 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b56dadd_0bce_4d6c_8041_f8dc3798e075.slice/crio-26fced9e78cec86a2421801593201b72c987cfd484431bada8cef9b0a5520250 WatchSource:0}: Error finding container 26fced9e78cec86a2421801593201b72c987cfd484431bada8cef9b0a5520250: Status 404 returned error can't find the container with id 26fced9e78cec86a2421801593201b72c987cfd484431bada8cef9b0a5520250 Jul 09 14:51:03.513515 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:03.513472 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" event={"ID":"4b56dadd-0bce-4d6c-8041-f8dc3798e075","Type":"ContainerStarted","Data":"073b36e7ff51d23aca4cc1733119b78731943cf2dfc02c9c1c846ec98a374000"} Jul 09 14:51:03.514020 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:03.513527 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" event={"ID":"4b56dadd-0bce-4d6c-8041-f8dc3798e075","Type":"ContainerStarted","Data":"26fced9e78cec86a2421801593201b72c987cfd484431bada8cef9b0a5520250"} Jul 09 14:51:03.534708 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:03.534656 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" podStartSLOduration=1.5346404 podStartE2EDuration="1.5346404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:51:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:51:03.532313902 +0000 UTC m=+2183.655453285" watchObservedRunningTime="2026-07-09 14:51:03.5346404 +0000 UTC m=+2183.657779787" Jul 09 14:51:06.837207 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:06.837166 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:51:12.707655 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:12.707618 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:12.710106 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:12.707670 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:12.720325 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:12.720296 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:13.579306 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:13.579264 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:51:32.136839 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:32.136806 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:51:46.405064 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:46.405021 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:51:53.864094 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:51:53.864062 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx"] Jul 09 14:52:07.244704 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:07.244619 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:52:08.884792 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:08.884749 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" podUID="4b56dadd-0bce-4d6c-8041-f8dc3798e075" containerName="main" containerID="cri-o://073b36e7ff51d23aca4cc1733119b78731943cf2dfc02c9c1c846ec98a374000" gracePeriod=45 Jul 09 14:52:09.141385 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.141315 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:52:09.247658 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.247612 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-home\") pod \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " Jul 09 14:52:09.247840 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.247685 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-dshm\") pod \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " Jul 09 14:52:09.247840 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.247729 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b56dadd-0bce-4d6c-8041-f8dc3798e075-tls-certs\") pod \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " Jul 09 14:52:09.247840 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.247748 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nm2r9\" (UniqueName: \"kubernetes.io/projected/4b56dadd-0bce-4d6c-8041-f8dc3798e075-kube-api-access-nm2r9\") pod \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " Jul 09 14:52:09.247840 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.247786 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-tmp-dir\") pod \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " Jul 09 14:52:09.247840 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.247833 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-model-cache\") pod \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\" (UID: \"4b56dadd-0bce-4d6c-8041-f8dc3798e075\") " Jul 09 14:52:09.248117 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.247890 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-home" (OuterVolumeSpecName: "home") pod "4b56dadd-0bce-4d6c-8041-f8dc3798e075" (UID: "4b56dadd-0bce-4d6c-8041-f8dc3798e075"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:52:09.248117 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.248081 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4b56dadd-0bce-4d6c-8041-f8dc3798e075" (UID: "4b56dadd-0bce-4d6c-8041-f8dc3798e075"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:52:09.248222 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.248161 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-model-cache" (OuterVolumeSpecName: "model-cache") pod "4b56dadd-0bce-4d6c-8041-f8dc3798e075" (UID: "4b56dadd-0bce-4d6c-8041-f8dc3798e075"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:52:09.248222 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.248183 2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:09.248222 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.248200 2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:09.249986 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.249941 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-dshm" (OuterVolumeSpecName: "dshm") pod "4b56dadd-0bce-4d6c-8041-f8dc3798e075" (UID: "4b56dadd-0bce-4d6c-8041-f8dc3798e075"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:52:09.249986 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.249974 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b56dadd-0bce-4d6c-8041-f8dc3798e075-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4b56dadd-0bce-4d6c-8041-f8dc3798e075" (UID: "4b56dadd-0bce-4d6c-8041-f8dc3798e075"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:52:09.249986 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.249979 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b56dadd-0bce-4d6c-8041-f8dc3798e075-kube-api-access-nm2r9" (OuterVolumeSpecName: "kube-api-access-nm2r9") pod "4b56dadd-0bce-4d6c-8041-f8dc3798e075" (UID: "4b56dadd-0bce-4d6c-8041-f8dc3798e075"). InnerVolumeSpecName "kube-api-access-nm2r9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:52:09.349299 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.349241 2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:09.349299 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.349289 2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b56dadd-0bce-4d6c-8041-f8dc3798e075-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:09.349299 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.349306 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nm2r9\" (UniqueName: \"kubernetes.io/projected/4b56dadd-0bce-4d6c-8041-f8dc3798e075-kube-api-access-nm2r9\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:09.349562 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.349318 2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b56dadd-0bce-4d6c-8041-f8dc3798e075-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:09.867653 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.867608 2581 generic.go:356] "Generic (PLEG): container finished" podID="4b56dadd-0bce-4d6c-8041-f8dc3798e075" containerID="073b36e7ff51d23aca4cc1733119b78731943cf2dfc02c9c1c846ec98a374000" exitCode=0 Jul 09 14:52:09.867840 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.867675 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" Jul 09 14:52:09.867840 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.867694 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" event={"ID":"4b56dadd-0bce-4d6c-8041-f8dc3798e075","Type":"ContainerDied","Data":"073b36e7ff51d23aca4cc1733119b78731943cf2dfc02c9c1c846ec98a374000"} Jul 09 14:52:09.867840 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.867744 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx" event={"ID":"4b56dadd-0bce-4d6c-8041-f8dc3798e075","Type":"ContainerDied","Data":"26fced9e78cec86a2421801593201b72c987cfd484431bada8cef9b0a5520250"} Jul 09 14:52:09.867840 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.867762 2581 scope.go:117] "RemoveContainer" containerID="073b36e7ff51d23aca4cc1733119b78731943cf2dfc02c9c1c846ec98a374000" Jul 09 14:52:09.877507 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.877486 2581 scope.go:117] "RemoveContainer" containerID="073b36e7ff51d23aca4cc1733119b78731943cf2dfc02c9c1c846ec98a374000" Jul 09 14:52:09.877751 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:52:09.877729 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"073b36e7ff51d23aca4cc1733119b78731943cf2dfc02c9c1c846ec98a374000\": container with ID starting with 073b36e7ff51d23aca4cc1733119b78731943cf2dfc02c9c1c846ec98a374000 not found: ID does not exist" containerID="073b36e7ff51d23aca4cc1733119b78731943cf2dfc02c9c1c846ec98a374000" Jul 09 14:52:09.877815 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.877763 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"073b36e7ff51d23aca4cc1733119b78731943cf2dfc02c9c1c846ec98a374000"} err="failed to get container status \"073b36e7ff51d23aca4cc1733119b78731943cf2dfc02c9c1c846ec98a374000\": rpc error: code = NotFound desc = could not find container \"073b36e7ff51d23aca4cc1733119b78731943cf2dfc02c9c1c846ec98a374000\": container with ID starting with 073b36e7ff51d23aca4cc1733119b78731943cf2dfc02c9c1c846ec98a374000 not found: ID does not exist" Jul 09 14:52:09.905089 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.905059 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx"] Jul 09 14:52:09.914134 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:09.914107 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c4eda7/rate-test-model-c4eda7-kserve-5d8c79c6b7-rc2wx"] Jul 09 14:52:10.977634 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:10.977597 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b56dadd-0bce-4d6c-8041-f8dc3798e075" path="/var/lib/kubelet/pods/4b56dadd-0bce-4d6c-8041-f8dc3798e075/volumes" Jul 09 14:52:18.719499 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:18.719461 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:52:28.443888 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:28.443855 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx"] Jul 09 14:52:28.444290 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:28.444130 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" podUID="586763e8-1619-4c6f-843b-f93b6b02bb4f" containerName="istio-proxy" containerID="cri-o://8b16b4b9e6b2ca431bd93bb263c3fb8b8a6a1d9530867df018b0623e45495328" gracePeriod=30 Jul 09 14:52:31.496580 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:31.496540 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zsj7t"] Jul 09 14:52:33.406747 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.406705 2581 patch_prober.go:28] interesting pod/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 14:52:33.407214 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.406796 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" podUID="586763e8-1619-4c6f-843b-f93b6b02bb4f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:52:33.703138 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.703109 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:52:33.797288 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.797249 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/586763e8-1619-4c6f-843b-f93b6b02bb4f-istiod-ca-cert\") pod \"586763e8-1619-4c6f-843b-f93b6b02bb4f\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " Jul 09 14:52:33.797476 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.797329 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-envoy\") pod \"586763e8-1619-4c6f-843b-f93b6b02bb4f\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " Jul 09 14:52:33.797476 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.797357 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-token\") pod \"586763e8-1619-4c6f-843b-f93b6b02bb4f\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " Jul 09 14:52:33.797476 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.797390 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-data\") pod \"586763e8-1619-4c6f-843b-f93b6b02bb4f\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " Jul 09 14:52:33.797476 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.797451 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-podinfo\") pod \"586763e8-1619-4c6f-843b-f93b6b02bb4f\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " Jul 09 14:52:33.797677 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.797488 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-workload-certs\") pod \"586763e8-1619-4c6f-843b-f93b6b02bb4f\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " Jul 09 14:52:33.797677 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.797515 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-workload-socket\") pod \"586763e8-1619-4c6f-843b-f93b6b02bb4f\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " Jul 09 14:52:33.797677 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.797549 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-772zg\" (UniqueName: \"kubernetes.io/projected/586763e8-1619-4c6f-843b-f93b6b02bb4f-kube-api-access-772zg\") pod \"586763e8-1619-4c6f-843b-f93b6b02bb4f\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " Jul 09 14:52:33.797677 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.797574 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-credential-socket\") pod \"586763e8-1619-4c6f-843b-f93b6b02bb4f\" (UID: \"586763e8-1619-4c6f-843b-f93b6b02bb4f\") " Jul 09 14:52:33.797881 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.797774 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "586763e8-1619-4c6f-843b-f93b6b02bb4f" (UID: "586763e8-1619-4c6f-843b-f93b6b02bb4f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:52:33.797881 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.797811 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "586763e8-1619-4c6f-843b-f93b6b02bb4f" (UID: "586763e8-1619-4c6f-843b-f93b6b02bb4f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:52:33.797881 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.797828 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/586763e8-1619-4c6f-843b-f93b6b02bb4f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "586763e8-1619-4c6f-843b-f93b6b02bb4f" (UID: "586763e8-1619-4c6f-843b-f93b6b02bb4f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:52:33.798154 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.798005 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "586763e8-1619-4c6f-843b-f93b6b02bb4f" (UID: "586763e8-1619-4c6f-843b-f93b6b02bb4f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:52:33.798154 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.798033 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/586763e8-1619-4c6f-843b-f93b6b02bb4f-istiod-ca-cert\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:33.798154 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.798055 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-workload-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:33.798154 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.798069 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-workload-socket\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:33.798570 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.798540 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-data" (OuterVolumeSpecName: "istio-data") pod "586763e8-1619-4c6f-843b-f93b6b02bb4f" (UID: "586763e8-1619-4c6f-843b-f93b6b02bb4f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:52:33.799687 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.799661 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "586763e8-1619-4c6f-843b-f93b6b02bb4f" (UID: "586763e8-1619-4c6f-843b-f93b6b02bb4f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:52:33.799801 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.799727 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "586763e8-1619-4c6f-843b-f93b6b02bb4f" (UID: "586763e8-1619-4c6f-843b-f93b6b02bb4f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:52:33.799864 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.799793 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-token" (OuterVolumeSpecName: "istio-token") pod "586763e8-1619-4c6f-843b-f93b6b02bb4f" (UID: "586763e8-1619-4c6f-843b-f93b6b02bb4f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:52:33.799922 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.799911 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/586763e8-1619-4c6f-843b-f93b6b02bb4f-kube-api-access-772zg" (OuterVolumeSpecName: "kube-api-access-772zg") pod "586763e8-1619-4c6f-843b-f93b6b02bb4f" (UID: "586763e8-1619-4c6f-843b-f93b6b02bb4f"). InnerVolumeSpecName "kube-api-access-772zg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:52:33.899387 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.899338 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-envoy\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:33.899387 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.899379 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-token\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:33.899387 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.899387 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-data\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:33.899387 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.899398 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/586763e8-1619-4c6f-843b-f93b6b02bb4f-istio-podinfo\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:33.899387 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.899407 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-772zg\" (UniqueName: \"kubernetes.io/projected/586763e8-1619-4c6f-843b-f93b6b02bb4f-kube-api-access-772zg\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:33.899707 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:33.899416 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/586763e8-1619-4c6f-843b-f93b6b02bb4f-credential-socket\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\"" Jul 09 14:52:34.006392 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.006301 2581 generic.go:356] "Generic (PLEG): container finished" podID="586763e8-1619-4c6f-843b-f93b6b02bb4f" containerID="8b16b4b9e6b2ca431bd93bb263c3fb8b8a6a1d9530867df018b0623e45495328" exitCode=0 Jul 09 14:52:34.006392 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.006355 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" Jul 09 14:52:34.006572 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.006388 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" event={"ID":"586763e8-1619-4c6f-843b-f93b6b02bb4f","Type":"ContainerDied","Data":"8b16b4b9e6b2ca431bd93bb263c3fb8b8a6a1d9530867df018b0623e45495328"} Jul 09 14:52:34.006572 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.006447 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx" event={"ID":"586763e8-1619-4c6f-843b-f93b6b02bb4f","Type":"ContainerDied","Data":"4154254d97507df0761ad6b96f0778e5a1ef358091ca97acade2b265406769b2"} Jul 09 14:52:34.006572 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.006478 2581 scope.go:117] "RemoveContainer" containerID="8b16b4b9e6b2ca431bd93bb263c3fb8b8a6a1d9530867df018b0623e45495328" Jul 09 14:52:34.017121 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.017102 2581 scope.go:117] "RemoveContainer" containerID="858231b5bbd658b1724ef9faf3f5e0648a648e192e93b159785bf02f2d9df91f" Jul 09 14:52:34.027294 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.027274 2581 scope.go:117] "RemoveContainer" containerID="8b16b4b9e6b2ca431bd93bb263c3fb8b8a6a1d9530867df018b0623e45495328" Jul 09 14:52:34.027554 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:52:34.027538 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b16b4b9e6b2ca431bd93bb263c3fb8b8a6a1d9530867df018b0623e45495328\": container with ID starting with 8b16b4b9e6b2ca431bd93bb263c3fb8b8a6a1d9530867df018b0623e45495328 not found: ID does not exist" containerID="8b16b4b9e6b2ca431bd93bb263c3fb8b8a6a1d9530867df018b0623e45495328" Jul 09 14:52:34.027602 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.027561 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b16b4b9e6b2ca431bd93bb263c3fb8b8a6a1d9530867df018b0623e45495328"} err="failed to get container status \"8b16b4b9e6b2ca431bd93bb263c3fb8b8a6a1d9530867df018b0623e45495328\": rpc error: code = NotFound desc = could not find container \"8b16b4b9e6b2ca431bd93bb263c3fb8b8a6a1d9530867df018b0623e45495328\": container with ID starting with 8b16b4b9e6b2ca431bd93bb263c3fb8b8a6a1d9530867df018b0623e45495328 not found: ID does not exist" Jul 09 14:52:34.027602 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.027577 2581 scope.go:117] "RemoveContainer" containerID="858231b5bbd658b1724ef9faf3f5e0648a648e192e93b159785bf02f2d9df91f" Jul 09 14:52:34.027800 ip-10-0-142-225 kubenswrapper[2581]: E0709 14:52:34.027775 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"858231b5bbd658b1724ef9faf3f5e0648a648e192e93b159785bf02f2d9df91f\": container with ID starting with 858231b5bbd658b1724ef9faf3f5e0648a648e192e93b159785bf02f2d9df91f not found: ID does not exist" containerID="858231b5bbd658b1724ef9faf3f5e0648a648e192e93b159785bf02f2d9df91f" Jul 09 14:52:34.027910 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.027802 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858231b5bbd658b1724ef9faf3f5e0648a648e192e93b159785bf02f2d9df91f"} err="failed to get container status \"858231b5bbd658b1724ef9faf3f5e0648a648e192e93b159785bf02f2d9df91f\": rpc error: code = NotFound desc = could not find container \"858231b5bbd658b1724ef9faf3f5e0648a648e192e93b159785bf02f2d9df91f\": container with ID starting with 858231b5bbd658b1724ef9faf3f5e0648a648e192e93b159785bf02f2d9df91f not found: ID does not exist" Jul 09 14:52:34.032081 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.032056 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx"] Jul 09 14:52:34.038029 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.038007 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-c4eda7-openshift-default-6b5659cfff-rvjzx"] Jul 09 14:52:34.942546 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.942516 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f8568cbd5-8nctl_ae8d4c12-0ee3-444d-ba0e-9f749336b750/copy-manifests/0.log" Jul 09 14:52:34.959084 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.959055 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f8568cbd5-8nctl_ae8d4c12-0ee3-444d-ba0e-9f749336b750/manager/0.log" Jul 09 14:52:34.977894 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:34.977862 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="586763e8-1619-4c6f-843b-f93b6b02bb4f" path="/var/lib/kubelet/pods/586763e8-1619-4c6f-843b-f93b6b02bb4f/volumes" Jul 09 14:52:36.295548 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:36.295520 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8_c27ee25b-9f01-4bc6-a509-67a61eaa1c1b/util/0.log" Jul 09 14:52:36.303961 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:36.303932 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8_c27ee25b-9f01-4bc6-a509-67a61eaa1c1b/pull/0.log" Jul 09 14:52:36.311055 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:36.311035 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8_c27ee25b-9f01-4bc6-a509-67a61eaa1c1b/extract/0.log" Jul 09 14:52:36.424755 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:36.424721 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j_51ca537b-213b-4d1f-aba5-c20dcc94d90d/util/0.log" Jul 09 14:52:36.431202 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:36.431178 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j_51ca537b-213b-4d1f-aba5-c20dcc94d90d/pull/0.log" Jul 09 14:52:36.438177 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:36.438153 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j_51ca537b-213b-4d1f-aba5-c20dcc94d90d/extract/0.log" Jul 09 14:52:36.549286 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:36.549159 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4_06ea3e13-372c-40f8-8c58-ce28e8fc64b6/extract/0.log" Jul 09 14:52:36.555698 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:36.555674 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4_06ea3e13-372c-40f8-8c58-ce28e8fc64b6/util/0.log" Jul 09 14:52:36.562282 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:36.562265 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4_06ea3e13-372c-40f8-8c58-ce28e8fc64b6/pull/0.log" Jul 09 14:52:36.677145 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:36.677113 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk_83e755e4-e8cd-4808-9c41-69afeee0f4f2/util/0.log" Jul 09 14:52:36.684409 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:36.684384 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk_83e755e4-e8cd-4808-9c41-69afeee0f4f2/pull/0.log" Jul 09 14:52:36.691357 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:36.691332 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk_83e755e4-e8cd-4808-9c41-69afeee0f4f2/extract/0.log" Jul 09 14:52:37.068637 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:37.068605 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-p7xtb_5f30310c-5fc0-4d28-b58f-0903c046070b/manager/0.log" Jul 09 14:52:37.540234 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:37.540198 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-zsj7t_3defbe4a-fb5a-4671-a840-0ee2b6c8e696/limitador/0.log" Jul 09 14:52:38.019263 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:38.019217 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46rvps_9b33aa88-9054-49e3-ba4d-14d98fa93454/istio-proxy/0.log" Jul 09 14:52:38.147258 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:38.147225 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b5v96_c40073d1-9c67-4907-b4d8-132c7b6dc4cc/discovery/0.log" Jul 09 14:52:38.510227 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:38.510187 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6sngn_34d9a37b-862d-4a2a-90f8-72c2e07cf9f9/istio-proxy/1.log" Jul 09 14:52:39.803520 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:39.803475 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-tbnhc_6bdce3eb-29c2-4681-ac86-12fdb98d8d9b/main/0.log" Jul 09 14:52:46.877838 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:46.877799 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5kc7s_6f9a1846-c394-4931-a7f5-d839e7558c70/global-pull-secret-syncer/0.log" Jul 09 14:52:47.152181 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:47.152098 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ghz28_bccdd948-f9a8-410b-ab36-404e262a2989/konnectivity-agent/0.log" Jul 09 14:52:47.237427 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:47.237396 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-225.ec2.internal_6bd8a30ab1f0a51bf21d4aa8ae4b5951/haproxy/0.log" Jul 09 14:52:51.776004 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:51.775970 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8_c27ee25b-9f01-4bc6-a509-67a61eaa1c1b/extract/0.log" Jul 09 14:52:51.807115 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:51.807079 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8_c27ee25b-9f01-4bc6-a509-67a61eaa1c1b/util/0.log" Jul 09 14:52:51.835734 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:51.835704 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c8fn8_c27ee25b-9f01-4bc6-a509-67a61eaa1c1b/pull/0.log" Jul 09 14:52:51.871199 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:51.871172 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j_51ca537b-213b-4d1f-aba5-c20dcc94d90d/extract/0.log" Jul 09 14:52:51.900832 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:51.900804 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j_51ca537b-213b-4d1f-aba5-c20dcc94d90d/util/0.log" Jul 09 14:52:51.931214 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:51.931186 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ljh9j_51ca537b-213b-4d1f-aba5-c20dcc94d90d/pull/0.log" Jul 09 14:52:51.973337 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:51.973304 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4_06ea3e13-372c-40f8-8c58-ce28e8fc64b6/extract/0.log" Jul 09 14:52:52.000128 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:52.000102 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4_06ea3e13-372c-40f8-8c58-ce28e8fc64b6/util/0.log" Jul 09 14:52:52.025717 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:52.025693 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737pns4_06ea3e13-372c-40f8-8c58-ce28e8fc64b6/pull/0.log" Jul 09 14:52:52.057441 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:52.057374 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk_83e755e4-e8cd-4808-9c41-69afeee0f4f2/extract/0.log" Jul 09 14:52:52.084653 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:52.084629 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk_83e755e4-e8cd-4808-9c41-69afeee0f4f2/util/0.log" Jul 09 14:52:52.114840 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:52.114810 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zktrk_83e755e4-e8cd-4808-9c41-69afeee0f4f2/pull/0.log" Jul 09 14:52:52.471381 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:52.471333 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-p7xtb_5f30310c-5fc0-4d28-b58f-0903c046070b/manager/0.log" Jul 09 14:52:52.712799 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:52.712769 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-zsj7t_3defbe4a-fb5a-4671-a840-0ee2b6c8e696/limitador/0.log" Jul 09 14:52:54.529840 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:54.529807 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a/alertmanager/0.log" Jul 09 14:52:54.557439 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:54.557412 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a/config-reloader/0.log" Jul 09 14:52:54.580709 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:54.580682 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a/kube-rbac-proxy-web/0.log" Jul 09 14:52:54.611274 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:54.611221 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a/kube-rbac-proxy/0.log" Jul 09 14:52:54.640304 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:54.640269 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a/kube-rbac-proxy-metric/0.log" Jul 09 14:52:54.665911 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:54.665889 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a/prom-label-proxy/0.log" Jul 09 14:52:54.693733 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:54.693705 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_571f0ace-8ae8-4040-8b3f-5a48e6c5bb3a/init-config-reloader/0.log" Jul 09 14:52:54.866015 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:54.865980 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6f87466b5b-72vqb_2ae7cac9-ee24-4331-aa0e-796fecabe79d/metrics-server/0.log" Jul 09 14:52:55.019081 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:55.019041 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8mkld_e7e93b15-3033-4616-9231-edd740d1f46b/node-exporter/0.log" Jul 09 14:52:55.044719 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:55.044691 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8mkld_e7e93b15-3033-4616-9231-edd740d1f46b/kube-rbac-proxy/0.log" Jul 09 14:52:55.068609 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:55.068585 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8mkld_e7e93b15-3033-4616-9231-edd740d1f46b/init-textfile/0.log" Jul 09 14:52:55.571941 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:55.571837 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7df788b5dd-qvh48_c77a677a-c1c4-477d-ae26-0b80b6ef2a2a/telemeter-client/0.log" Jul 09 14:52:55.636831 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:55.636796 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7df788b5dd-qvh48_c77a677a-c1c4-477d-ae26-0b80b6ef2a2a/reload/0.log" Jul 09 14:52:55.673077 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:55.673048 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7df788b5dd-qvh48_c77a677a-c1c4-477d-ae26-0b80b6ef2a2a/kube-rbac-proxy/0.log" Jul 09 14:52:55.723179 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:55.723146 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d5dccd9cc-5tb7k_cd9d70aa-c3ff-4be6-9508-e73c33a43e98/thanos-query/0.log" Jul 09 14:52:55.749751 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:55.749723 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d5dccd9cc-5tb7k_cd9d70aa-c3ff-4be6-9508-e73c33a43e98/kube-rbac-proxy-web/0.log" Jul 09 14:52:55.775107 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:55.775083 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d5dccd9cc-5tb7k_cd9d70aa-c3ff-4be6-9508-e73c33a43e98/kube-rbac-proxy/0.log" Jul 09 14:52:55.801727 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:55.801703 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d5dccd9cc-5tb7k_cd9d70aa-c3ff-4be6-9508-e73c33a43e98/prom-label-proxy/0.log" Jul 09 14:52:55.836373 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:55.836261 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d5dccd9cc-5tb7k_cd9d70aa-c3ff-4be6-9508-e73c33a43e98/kube-rbac-proxy-rules/0.log" Jul 09 14:52:55.862617 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:55.862594 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d5dccd9cc-5tb7k_cd9d70aa-c3ff-4be6-9508-e73c33a43e98/kube-rbac-proxy-metrics/0.log" Jul 09 14:52:56.705537 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.705499 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf"] Jul 09 14:52:56.705929 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.705878 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="586763e8-1619-4c6f-843b-f93b6b02bb4f" containerName="istio-proxy" Jul 09 14:52:56.705929 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.705890 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="586763e8-1619-4c6f-843b-f93b6b02bb4f" containerName="istio-proxy" Jul 09 14:52:56.705929 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.705900 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="586763e8-1619-4c6f-843b-f93b6b02bb4f" containerName="istio-proxy" Jul 09 14:52:56.705929 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.705906 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="586763e8-1619-4c6f-843b-f93b6b02bb4f" containerName="istio-proxy" Jul 09 14:52:56.705929 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.705918 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b56dadd-0bce-4d6c-8041-f8dc3798e075" containerName="main" Jul 09 14:52:56.705929 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.705924 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b56dadd-0bce-4d6c-8041-f8dc3798e075" containerName="main" Jul 09 14:52:56.706230 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.705995 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="586763e8-1619-4c6f-843b-f93b6b02bb4f" containerName="istio-proxy" Jul 09 14:52:56.706230 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.706005 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="586763e8-1619-4c6f-843b-f93b6b02bb4f" containerName="istio-proxy" Jul 09 14:52:56.706230 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.706010 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b56dadd-0bce-4d6c-8041-f8dc3798e075" containerName="main" Jul 09 14:52:56.709219 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.709195 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.712432 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.712400 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mpppr\"/\"default-dockercfg-mbnb2\"" Jul 09 14:52:56.712572 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.712462 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mpppr\"/\"kube-root-ca.crt\"" Jul 09 14:52:56.713726 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.713709 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mpppr\"/\"openshift-service-ca.crt\"" Jul 09 14:52:56.719302 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.719279 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf"] Jul 09 14:52:56.818084 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.818050 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-podres\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.818292 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.818109 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-lib-modules\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.818292 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.818181 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-proc\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.818292 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.818285 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n96m\" (UniqueName: \"kubernetes.io/projected/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-kube-api-access-5n96m\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.818425 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.818311 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-sys\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.919166 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.919123 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-podres\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.919390 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.919177 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-lib-modules\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.919390 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.919203 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-proc\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.919390 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.919276 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n96m\" (UniqueName: \"kubernetes.io/projected/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-kube-api-access-5n96m\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.919390 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.919304 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-podres\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.919390 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.919332 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-proc\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.919390 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.919363 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-sys\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.919390 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.919383 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-lib-modules\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.919712 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.919434 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-sys\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:56.928355 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:56.928326 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n96m\" (UniqueName: \"kubernetes.io/projected/09c6ca0f-0d81-48a6-b1e1-ceb65e87b092-kube-api-access-5n96m\") pod \"perf-node-gather-daemonset-57psf\" (UID: \"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092\") " pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:57.020195 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:57.020099 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:57.152930 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:57.152903 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf"] Jul 09 14:52:57.155106 ip-10-0-142-225 kubenswrapper[2581]: W0709 14:52:57.155075 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod09c6ca0f_0d81_48a6_b1e1_ceb65e87b092.slice/crio-12b9e9ae1eb25dd99e6a8738ae383ec39e27a9291599582417f2746a1fef5102 WatchSource:0}: Error finding container 12b9e9ae1eb25dd99e6a8738ae383ec39e27a9291599582417f2746a1fef5102: Status 404 returned error can't find the container with id 12b9e9ae1eb25dd99e6a8738ae383ec39e27a9291599582417f2746a1fef5102 Jul 09 14:52:58.132200 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:58.132163 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" event={"ID":"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092","Type":"ContainerStarted","Data":"e9c9087d368c53f86d182a4a5c16b458cf64ade7c0e0d926f3bfc5629792641b"} Jul 09 14:52:58.132567 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:58.132217 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" event={"ID":"09c6ca0f-0d81-48a6-b1e1-ceb65e87b092","Type":"ContainerStarted","Data":"12b9e9ae1eb25dd99e6a8738ae383ec39e27a9291599582417f2746a1fef5102"} Jul 09 14:52:58.132567 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:58.132253 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:52:58.150594 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:58.150542 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" podStartSLOduration=2.150525093 podStartE2EDuration="2.150525093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:52:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:52:58.150026013 +0000 UTC m=+2298.273165417" watchObservedRunningTime="2026-07-09 14:52:58.150525093 +0000 UTC m=+2298.273664476" Jul 09 14:52:58.177063 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:58.177031 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7f569c684-cmlq7_30fd575c-e61c-4ff2-9608-4385bd0bcb4b/console/0.log" Jul 09 14:52:59.987055 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:52:59.987015 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mpgmz_9a0409b2-5366-4b47-9fd8-1f893c1d43d0/dns/0.log" Jul 09 14:53:00.011112 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:00.011085 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mpgmz_9a0409b2-5366-4b47-9fd8-1f893c1d43d0/kube-rbac-proxy/0.log" Jul 09 14:53:00.095120 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:00.095088 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-thcjx_4d486c18-e090-48ce-bc55-8d752af57ea6/dns-node-resolver/0.log" Jul 09 14:53:00.658808 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:00.658775 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-9f8cc8f45-kvxn7_741c6816-f1eb-478a-a1f2-f77d86cf08d3/registry/0.log" Jul 09 14:53:00.737447 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:00.737418 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xj9gv_4ff0d36b-2274-490a-9421-62b5a55e6028/node-ca/0.log" Jul 09 14:53:01.720770 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:01.720736 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46rvps_9b33aa88-9054-49e3-ba4d-14d98fa93454/istio-proxy/0.log" Jul 09 14:53:01.830946 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:01.830912 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b5v96_c40073d1-9c67-4907-b4d8-132c7b6dc4cc/discovery/0.log" Jul 09 14:53:01.913099 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:01.913063 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6sngn_34d9a37b-862d-4a2a-90f8-72c2e07cf9f9/istio-proxy/0.log" Jul 09 14:53:01.938187 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:01.938155 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6sngn_34d9a37b-862d-4a2a-90f8-72c2e07cf9f9/istio-proxy/1.log" Jul 09 14:53:02.780697 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:02.780669 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kksh7_a3216d15-42b7-49e7-b5b8-01f35726265b/serve-healthcheck-canary/0.log" Jul 09 14:53:03.366844 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:03.366814 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9zzdz_a4551179-366a-4b3a-a07e-12e4d2d32994/kube-rbac-proxy/0.log" Jul 09 14:53:03.390521 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:03.390493 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9zzdz_a4551179-366a-4b3a-a07e-12e4d2d32994/exporter/0.log" Jul 09 14:53:03.413942 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:03.413915 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9zzdz_a4551179-366a-4b3a-a07e-12e4d2d32994/extractor/0.log" Jul 09 14:53:04.148087 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:04.148062 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mpppr/perf-node-gather-daemonset-57psf" Jul 09 14:53:06.463348 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:06.463314 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f8568cbd5-8nctl_ae8d4c12-0ee3-444d-ba0e-9f749336b750/manager/0.log" Jul 09 14:53:06.482892 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:06.482866 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f8568cbd5-8nctl_ae8d4c12-0ee3-444d-ba0e-9f749336b750/copy-manifests/0.log" Jul 09 14:53:07.439260 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:07.439226 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46rvps_9b33aa88-9054-49e3-ba4d-14d98fa93454/istio-proxy/0.log" Jul 09 14:53:07.549468 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:07.549427 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b5v96_c40073d1-9c67-4907-b4d8-132c7b6dc4cc/discovery/0.log" Jul 09 14:53:07.606440 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:07.606413 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6sngn_34d9a37b-862d-4a2a-90f8-72c2e07cf9f9/istio-proxy/0.log" Jul 09 14:53:07.632090 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:07.632061 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6sngn_34d9a37b-862d-4a2a-90f8-72c2e07cf9f9/istio-proxy/1.log" Jul 09 14:53:07.797994 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:07.797939 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5578c64b55-q78qs_4997f2ce-3bc8-4baf-84f9-82728106d0cf/manager/0.log" Jul 09 14:53:07.830207 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:07.830181 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rkd5n_1030ebb6-047f-4a4a-89b4-ea08fc0fb8fc/openshift-lws-operator/0.log" Jul 09 14:53:09.030741 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:09.030709 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46rvps_9b33aa88-9054-49e3-ba4d-14d98fa93454/istio-proxy/0.log" Jul 09 14:53:09.125278 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:09.125246 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b5v96_c40073d1-9c67-4907-b4d8-132c7b6dc4cc/discovery/0.log" Jul 09 14:53:09.250670 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:09.250640 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6sngn_34d9a37b-862d-4a2a-90f8-72c2e07cf9f9/istio-proxy/0.log" Jul 09 14:53:09.283074 ip-10-0-142-225 kubenswrapper[2581]: I0709 14:53:09.282998 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6sngn_34d9a37b-862d-4a2a-90f8-72c2e07cf9f9/istio-proxy/1.log"