Jul 12 18:25:08.663262 ip-10-0-140-170 systemd[1]: Starting Kubernetes Kubelet... Jul 12 18:25:09.157793 ip-10-0-140-170 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 18:25:09.157793 ip-10-0-140-170 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 12 18:25:09.157793 ip-10-0-140-170 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 18:25:09.157793 ip-10-0-140-170 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 12 18:25:09.157793 ip-10-0-140-170 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 18:25:09.157793 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.120803 2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 12 18:25:09.157793 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.125027 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 18:25:09.157793 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.125042 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 18:25:09.157793 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.125046 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 18:25:09.157793 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.125050 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 18:25:09.157793 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.125053 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 18:25:09.157793 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.125056 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 18:25:09.157793 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.125060 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 18:25:09.157793 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.125063 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.125066 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.125069 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.125072 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.125132 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127093 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127103 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127108 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127111 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127114 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127118 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127121 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127124 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127127 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127130 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127134 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127137 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127140 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127147 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127150 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 18:25:09.160207 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127153 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127157 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127164 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127170 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127174 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127179 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127182 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127186 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127189 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127193 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127199 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127203 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127206 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127209 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127212 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127225 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127228 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127231 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127234 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 18:25:09.162658 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127237 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127242 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127246 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127252 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127255 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127259 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127261 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127264 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127268 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127271 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127274 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127277 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127280 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127283 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127286 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127319 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127342 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127348 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127353 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127358 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 18:25:09.164520 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127363 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127372 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127379 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127382 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127390 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127395 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127402 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127405 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127408 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127411 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127414 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127417 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127422 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127424 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127427 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127430 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127433 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127437 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127440 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 18:25:09.166853 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127445 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127449 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127452 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127455 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127458 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127460 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127463 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127466 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127469 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127473 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127476 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127478 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127484 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.127487 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128468 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128479 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128482 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128485 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128488 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128491 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 18:25:09.181514 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128494 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128497 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128500 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128502 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128505 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128508 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128511 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128513 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128516 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128519 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128522 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128525 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128528 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128532 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128535 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128537 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128541 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128543 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128546 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 18:25:09.184102 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128549 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128553 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128557 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128561 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128564 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128568 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128571 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128574 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128578 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128581 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128584 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128586 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128589 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128593 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128596 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128599 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128602 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128605 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128608 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 18:25:09.185941 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128611 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128614 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128616 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128619 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128626 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128629 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128632 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128635 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128638 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128640 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128643 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128646 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128649 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128651 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128654 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128656 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128659 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128662 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128665 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128667 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 18:25:09.201901 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128670 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128672 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128675 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128677 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128681 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128685 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128688 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128690 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128693 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128696 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128698 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128704 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128707 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128710 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128713 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128716 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128718 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128720 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128723 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 18:25:09.203894 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128726 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128728 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128730 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128733 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128735 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128738 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128741 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128743 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128746 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128749 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128751 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128755 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128757 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128760 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128762 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.128765 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129221 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129229 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129232 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129234 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 18:25:09.206569 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129237 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129240 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129243 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129245 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129248 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129250 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129253 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129256 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129259 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129261 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129264 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129266 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129270 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129272 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129275 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129278 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129281 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129285 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129287 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129290 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 18:25:09.213526 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129293 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129296 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129298 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129301 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129304 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129307 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129309 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129312 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129314 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129317 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129319 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129322 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129324 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129327 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129329 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129334 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129337 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129341 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129344 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 18:25:09.216685 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129346 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 18:25:09.213993 ip-10-0-140-170 systemd[1]: Started Kubernetes Kubelet. Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129349 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129352 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129355 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129358 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129361 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129364 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129367 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129371 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129373 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129376 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129379 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129381 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129385 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129389 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129392 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129394 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129397 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129400 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129402 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 18:25:09.229822 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129405 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129407 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129410 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129412 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129415 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129418 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129420 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129423 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129425 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129428 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129430 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129433 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129435 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129439 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129443 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129445 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129448 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129450 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129452 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129455 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 18:25:09.246122 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129458 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129461 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129465 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129468 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129471 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129474 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129476 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129482 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129484 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129487 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129491 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129493 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129496 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129499 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129501 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.129504 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129584 2585 flags.go:64] FLAG: --address="0.0.0.0" Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129593 2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129600 2585 flags.go:64] FLAG: --anonymous-auth="true" Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129604 2585 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129609 2585 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 12 18:25:09.261788 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129612 2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129616 2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129620 2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129624 2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129627 2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129631 2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129635 2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129638 2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129641 2585 flags.go:64] FLAG: --cgroup-root="" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129643 2585 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129647 2585 flags.go:64] FLAG: --client-ca-file="" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129650 2585 flags.go:64] FLAG: --cloud-provider="external" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129653 2585 flags.go:64] FLAG: --cluster-dns="[]" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129657 2585 flags.go:64] FLAG: --cluster-domain="" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129660 2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129663 2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129666 2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129669 2585 flags.go:64] FLAG: --container-log-max-files="5" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129676 2585 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129679 2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129683 2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129686 2585 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129690 2585 flags.go:64] FLAG: --contention-profiling="false" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129693 2585 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 12 18:25:09.407302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129696 2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129699 2585 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129702 2585 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129706 2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129709 2585 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129712 2585 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129714 2585 flags.go:64] FLAG: --enable-load-reader="false" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129718 2585 flags.go:64] FLAG: --enable-server="true" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129720 2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129727 2585 flags.go:64] FLAG: --event-burst="100" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129730 2585 flags.go:64] FLAG: --event-qps="50" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129733 2585 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129736 2585 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129739 2585 flags.go:64] FLAG: --eviction-hard="" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129743 2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129746 2585 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129749 2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129751 2585 flags.go:64] FLAG: --eviction-soft="" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129754 2585 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129757 2585 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129760 2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129763 2585 flags.go:64] FLAG: --experimental-mounter-path="" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129766 2585 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129769 2585 flags.go:64] FLAG: --fail-swap-on="true" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129771 2585 flags.go:64] FLAG: --feature-gates="" Jul 12 18:25:09.429836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129775 2585 flags.go:64] FLAG: --file-check-frequency="20s" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129779 2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129782 2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129786 2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129789 2585 flags.go:64] FLAG: --healthz-port="10248" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129792 2585 flags.go:64] FLAG: --help="false" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129795 2585 flags.go:64] FLAG: --hostname-override="ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129798 2585 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129801 2585 flags.go:64] FLAG: --http-check-frequency="20s" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129804 2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129807 2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129811 2585 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129813 2585 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129816 2585 flags.go:64] FLAG: --image-service-endpoint="" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129819 2585 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129822 2585 flags.go:64] FLAG: --kube-api-burst="100" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129826 2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129829 2585 flags.go:64] FLAG: --kube-api-qps="50" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129832 2585 flags.go:64] FLAG: --kube-reserved="" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129835 2585 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129838 2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129840 2585 flags.go:64] FLAG: --kubelet-cgroups="" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129843 2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129846 2585 flags.go:64] FLAG: --lock-file="" Jul 12 18:25:09.431918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129849 2585 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129852 2585 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129855 2585 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129860 2585 flags.go:64] FLAG: --log-json-split-stream="false" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129863 2585 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129865 2585 flags.go:64] FLAG: --log-text-split-stream="false" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129868 2585 flags.go:64] FLAG: --logging-format="text" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129871 2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129874 2585 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129879 2585 flags.go:64] FLAG: --manifest-url="" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129882 2585 flags.go:64] FLAG: --manifest-url-header="" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129886 2585 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129889 2585 flags.go:64] FLAG: --max-open-files="1000000" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129894 2585 flags.go:64] FLAG: --max-pods="110" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129897 2585 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129900 2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129903 2585 flags.go:64] FLAG: --memory-manager-policy="None" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129906 2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129909 2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129912 2585 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129915 2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129922 2585 flags.go:64] FLAG: --node-status-max-images="50" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129925 2585 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129943 2585 flags.go:64] FLAG: --oom-score-adj="-999" Jul 12 18:25:09.440350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129949 2585 flags.go:64] FLAG: --pod-cidr="" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129952 2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129958 2585 flags.go:64] FLAG: --pod-manifest-path="" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129961 2585 flags.go:64] FLAG: --pod-max-pids="-1" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129964 2585 flags.go:64] FLAG: --pods-per-core="0" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129966 2585 flags.go:64] FLAG: --port="10250" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129970 2585 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129973 2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0524f4ded4cf920e0" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129976 2585 flags.go:64] FLAG: --qos-reserved="" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129979 2585 flags.go:64] FLAG: --read-only-port="10255" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129982 2585 flags.go:64] FLAG: --register-node="true" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129985 2585 flags.go:64] FLAG: --register-with-taints="" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129989 2585 flags.go:64] FLAG: --registry-burst="10" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129992 2585 flags.go:64] FLAG: --registry-qps="5" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129994 2585 flags.go:64] FLAG: --reserved-cpus="" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.129997 2585 flags.go:64] FLAG: --reserved-memory="" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130006 2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130009 2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130013 2585 flags.go:64] FLAG: --rotate-certificates="false" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130016 2585 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130019 2585 flags.go:64] FLAG: --runonce="false" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130022 2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130025 2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130028 2585 flags.go:64] FLAG: --seccomp-default="false" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130031 2585 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 12 18:25:09.441203 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130034 2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130037 2585 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130040 2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130044 2585 flags.go:64] FLAG: --storage-driver-password="root" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130047 2585 flags.go:64] FLAG: --storage-driver-secure="false" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130049 2585 flags.go:64] FLAG: --storage-driver-table="stats" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130052 2585 flags.go:64] FLAG: --storage-driver-user="root" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130056 2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130059 2585 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130062 2585 flags.go:64] FLAG: --system-cgroups="" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130065 2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130070 2585 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130073 2585 flags.go:64] FLAG: --tls-cert-file="" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130076 2585 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130080 2585 flags.go:64] FLAG: --tls-min-version="" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130082 2585 flags.go:64] FLAG: --tls-private-key-file="" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130085 2585 flags.go:64] FLAG: --topology-manager-policy="none" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130088 2585 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130091 2585 flags.go:64] FLAG: --topology-manager-scope="container" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130094 2585 flags.go:64] FLAG: --v="2" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130097 2585 flags.go:64] FLAG: --version="false" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130101 2585 flags.go:64] FLAG: --vmodule="" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130105 2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.130108 2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130245 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 18:25:09.443892 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130250 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130254 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130257 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130260 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130262 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130265 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130268 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130271 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130274 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130278 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130282 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130285 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130288 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130290 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130294 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130297 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130300 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130302 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130305 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130307 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 18:25:09.444704 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130310 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130312 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130315 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130317 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130320 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130322 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130325 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130328 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130330 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130333 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130335 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130338 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130342 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130346 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130348 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130351 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130354 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130356 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130359 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130361 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 18:25:09.445428 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130364 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130367 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130370 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130372 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130375 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130377 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130381 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130384 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130386 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130389 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130391 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130394 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130397 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130400 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130402 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130405 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130407 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130410 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130412 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 18:25:09.446192 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130415 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130417 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130420 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130422 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130425 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130428 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130431 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130436 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130439 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130442 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130445 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130447 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130450 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130453 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130456 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130459 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130461 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130464 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130466 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 18:25:09.446817 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130470 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130473 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130475 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130478 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130480 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130483 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130485 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130488 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130490 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130493 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130497 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130500 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130502 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130505 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130507 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130510 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130512 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130515 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130519 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 18:25:09.447446 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.130521 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.131304 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.139953 2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.139970 2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.139988 2585 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.139994 2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140061 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140067 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140070 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140074 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140077 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140080 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140083 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140086 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140088 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 18:25:09.448110 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140091 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140094 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140097 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140099 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140102 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140105 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140107 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140110 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140113 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140115 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140118 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140121 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140124 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140126 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140129 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140132 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140135 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140139 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140143 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140147 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 18:25:09.448592 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140151 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140154 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140157 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140160 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140163 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140165 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140168 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140171 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140173 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140176 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140178 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140181 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140184 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140187 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140189 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140192 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140194 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140197 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140200 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140202 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 18:25:09.449297 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140205 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140208 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140211 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140213 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140216 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140219 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140222 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140224 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140228 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140231 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140234 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140238 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140241 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140245 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140248 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140251 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140254 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140257 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140260 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140262 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 18:25:09.608615 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140266 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140268 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140271 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140274 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140276 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140279 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140281 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140284 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140286 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140289 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140291 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140294 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140296 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140299 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140301 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140304 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140307 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140309 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140311 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 18:25:09.609369 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140314 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140319 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140322 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140324 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140327 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140329 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140332 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140388 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140394 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140397 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.140400 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.140406 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141926 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141943 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141947 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 18:25:09.610027 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141950 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141953 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141956 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141959 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141961 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141964 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141967 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141970 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141972 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141977 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141980 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141982 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141985 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141988 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141990 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141993 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141995 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.141998 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142002 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142005 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 18:25:09.610458 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142008 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142010 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142013 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142016 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142019 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142028 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142031 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142033 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142036 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142038 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142041 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142043 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142046 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142048 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142051 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142054 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142056 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142061 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142064 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142067 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 18:25:09.611098 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142070 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142074 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142077 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142079 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142083 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142086 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142089 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142091 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142094 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142113 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142317 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142319 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142322 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142325 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142327 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142330 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142332 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142539 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142548 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 18:25:09.611664 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142551 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142758 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142761 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142764 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142766 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142975 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142978 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142981 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142983 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142986 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142988 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142991 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.142993 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143013 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143018 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143021 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143024 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143027 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143030 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143033 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 18:25:09.612254 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143036 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143039 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143041 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143045 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143048 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143051 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143053 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143056 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143059 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143061 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143064 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143068 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143078 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143081 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143084 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143086 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 18:25:09.612857 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:09.143089 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 18:25:09.613377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.143094 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 18:25:09.613377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.143249 2585 server.go:962] "Client rotation is on, will bootstrap in background" Jul 12 18:25:09.613377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.148244 2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 12 18:25:09.613377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.150064 2585 server.go:1019] "Starting client certificate rotation" Jul 12 18:25:09.613377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.150182 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 18:25:09.613377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.150234 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 18:25:09.613377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.153781 2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 18:25:09.613377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.155356 2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 18:25:09.613377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.170290 2585 log.go:25] "Validated CRI v1 runtime API" Jul 12 18:25:09.613377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.173423 2585 log.go:25] "Validated CRI v1 image API" Jul 12 18:25:09.613377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.174478 2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 12 18:25:09.613377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.177140 2585 fs.go:135] Filesystem UUIDs: map[57bf0e67-84b6-429e-aa97-0108dcd010b5:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bbfb9de2-a600-4c9f-953a-0be921ef038c:/dev/nvme0n1p3] Jul 12 18:25:09.613747 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.177157 2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 12 18:25:09.613747 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.182290 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.185415 2585 manager.go:216] Machine: {Timestamp:2026-07-12 18:25:09.182948917 +0000 UTC m=+0.382415939 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098877 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fa0a02a5774b784720329d2bbffbe SystemUUID:ec2fa0a0-2a57-74b7-8472-0329d2bbffbe BootID:5dd6c48d-4331-4d28-ab2b-bf1072a9552e Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2e:40:35:07:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2e:40:35:07:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:d0:cd:f4:06:f5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.185521 2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.185612 2585 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.190021 2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.190056 2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-170.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.190898 2585 topology_manager.go:138] "Creating topology manager with none policy" Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.190909 2585 container_manager_linux.go:309] "Creating device plugin manager" Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.190917 2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.191710 2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.193450 2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.197018 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.197145 2585 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.197174 2585 kubelet.go:486] "Attempting to sync node with API server" Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.197184 2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.197201 2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.197213 2585 kubelet.go:398] "Adding apiserver pod source" Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.197223 2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 12 18:25:09.614414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.197595 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.197611 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.200100 2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.203741 2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.203779 2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.204047 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.204068 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.204080 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.204090 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.204105 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.204117 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.204590 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.204602 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.204617 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.204626 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.204638 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.204648 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.204680 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.204687 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.208858 2585 server.go:1268] "Started kubelet" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.208969 2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.209007 2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.209096 2585 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.213738 2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-170.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.213792 2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.214033 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-170.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.214181 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.218246 2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.218347 2585 server.go:310] "Adding debug handlers to kubelet server" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.220890 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-170.ec2.internal.18c19e0dba1076fb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-170.ec2.internal,UID:ip-10-0-140-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-170.ec2.internal,},FirstTimestamp:2026-07-12 18:25:09.208831739 +0000 UTC m=+0.408298760,LastTimestamp:2026-07-12 18:25:09.208831739 +0000 UTC m=+0.408298760,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-170.ec2.internal,}" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.224694 2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.224715 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.224800 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.225525 2585 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.225540 2585 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.225525 2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.225695 2585 reconstruct.go:97] "Volume reconstruction finished" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.225703 2585 reconciler.go:29] "Reconciler: start to sync state" Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.226333 2585 factory.go:55] Registering systemd factory Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.226345 2585 factory.go:223] Registration of the systemd container factory successfully Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.226545 2585 factory.go:153] Registering CRI-O factory Jul 12 18:25:09.615010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.226556 2585 factory.go:223] Registration of the crio container factory successfully Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.226622 2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.226647 2585 factory.go:103] Registering Raw factory Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.226662 2585 manager.go:1196] Started watching for new ooms in manager Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.227167 2585 manager.go:349] Starting recovery of all containers Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.231652 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.231721 2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-170.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.236475 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-htmbj" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.241275 2585 manager.go:354] Recovery completed Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.242554 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-htmbj" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.246289 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.252093 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasSufficientMemory" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.252121 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.252135 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasSufficientPID" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.252709 2585 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.252719 2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.252725 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.252734 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.253668 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasSufficientMemory" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.253699 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.253715 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasSufficientPID" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.254018 2585 policy_none.go:49] "None policy: Start" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.254035 2585 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.254049 2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.254112 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-170.ec2.internal.18c19e0dbca4c3db default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-170.ec2.internal,UID:ip-10-0-140-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-170.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-170.ec2.internal,},FirstTimestamp:2026-07-12 18:25:09.252105179 +0000 UTC m=+0.451572200,LastTimestamp:2026-07-12 18:25:09.252105179 +0000 UTC m=+0.451572200,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-170.ec2.internal,}" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.255027 2585 policy_none.go:47] "Start" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.296605 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.297827 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.297874 2585 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.297900 2585 kubelet.go:2448] "Starting kubelet main sync loop" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.297957 2585 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.300808 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.302666 2585 manager.go:337] "Starting Device Plugin manager" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.302702 2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.302712 2585 server.go:87] "Starting device plugin registration server" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.303000 2585 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.303011 2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.303125 2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.303227 2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 12 18:25:09.800812 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.303238 2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.303704 2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.303749 2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.398870 2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-170.ec2.internal"] Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.398987 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.400051 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasSufficientMemory" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.400077 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.400089 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasSufficientPID" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.401347 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.401464 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.401492 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.402150 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasSufficientMemory" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.402175 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.402188 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasSufficientPID" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.402308 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasSufficientMemory" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.402337 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.402350 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasSufficientPID" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.403132 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.403428 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.403450 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.404492 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasSufficientMemory" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.404520 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.404535 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasSufficientPID" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.404562 2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.404521 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasSufficientMemory" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.404672 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.404686 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeHasSufficientPID" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.412862 2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.412881 2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-170.ec2.internal\": node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.427153 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b1c54e9c324d378d37710ac8849ecf53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal\" (UID: \"b1c54e9c324d378d37710ac8849ecf53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.427178 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1c54e9c324d378d37710ac8849ecf53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal\" (UID: \"b1c54e9c324d378d37710ac8849ecf53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.427195 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ec39bd46f4e845dad75a1dba09eb9d6f-config\") pod \"kube-apiserver-proxy-ip-10-0-140-170.ec2.internal\" (UID: \"ec39bd46f4e845dad75a1dba09eb9d6f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.435762 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:09.802653 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.446895 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-170.ec2.internal\" not found" node="ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.803554 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.451603 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-170.ec2.internal\" not found" node="ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.803554 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.527878 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b1c54e9c324d378d37710ac8849ecf53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal\" (UID: \"b1c54e9c324d378d37710ac8849ecf53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.803554 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.527908 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1c54e9c324d378d37710ac8849ecf53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal\" (UID: \"b1c54e9c324d378d37710ac8849ecf53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.803554 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.527924 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ec39bd46f4e845dad75a1dba09eb9d6f-config\") pod \"kube-apiserver-proxy-ip-10-0-140-170.ec2.internal\" (UID: \"ec39bd46f4e845dad75a1dba09eb9d6f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.803554 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.527988 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b1c54e9c324d378d37710ac8849ecf53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal\" (UID: \"b1c54e9c324d378d37710ac8849ecf53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.803554 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.527998 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1c54e9c324d378d37710ac8849ecf53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal\" (UID: \"b1c54e9c324d378d37710ac8849ecf53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.803554 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.528061 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ec39bd46f4e845dad75a1dba09eb9d6f-config\") pod \"kube-apiserver-proxy-ip-10-0-140-170.ec2.internal\" (UID: \"ec39bd46f4e845dad75a1dba09eb9d6f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.803554 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.536002 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:09.803554 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.636585 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:09.803554 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.737381 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:09.803554 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.749584 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal" Jul 12 18:25:09.803554 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:09.755774 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-170.ec2.internal" Jul 12 18:25:10.017742 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.838405 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:10.017742 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:09.939142 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:10.229380 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:10.040185 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:10.229380 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:10.140766 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:10.229380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:10.149617 2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 12 18:25:10.229380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:10.149777 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 18:25:10.386863 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:10.241800 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:10.386863 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:10.246224 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 18:20:09 +0000 UTC" deadline="2028-03-28 23:30:59.06231348 +0000 UTC" Jul 12 18:25:10.386863 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:10.246258 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15005h5m48.816059093s" Jul 12 18:25:10.386863 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:10.342688 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:10.545719 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:10.443384 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:10.545719 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:10.463317 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 18:25:10.545719 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:10.544200 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:10.694259 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:10.564115 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec39bd46f4e845dad75a1dba09eb9d6f.slice/crio-766cdc02dc399aa7fe1cfdb9ab46d820f7376f6a9771332656b7eb24d076e0dc WatchSource:0}: Error finding container 766cdc02dc399aa7fe1cfdb9ab46d820f7376f6a9771332656b7eb24d076e0dc: Status 404 returned error can't find the container with id 766cdc02dc399aa7fe1cfdb9ab46d820f7376f6a9771332656b7eb24d076e0dc Jul 12 18:25:10.694259 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:10.564430 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1c54e9c324d378d37710ac8849ecf53.slice/crio-813b2f6a76b6d9c9ded60e3f248a372b05b94cf8b2230d5e92fe6de2a32bd867 WatchSource:0}: Error finding container 813b2f6a76b6d9c9ded60e3f248a372b05b94cf8b2230d5e92fe6de2a32bd867: Status 404 returned error can't find the container with id 813b2f6a76b6d9c9ded60e3f248a372b05b94cf8b2230d5e92fe6de2a32bd867 Jul 12 18:25:10.694259 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:10.569331 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 18:25:10.694259 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:10.644925 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:10.694259 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:10.690129 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 18:25:10.873284 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:10.745307 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-170.ec2.internal\" not found" Jul 12 18:25:10.873284 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:10.789455 2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 18:25:10.873284 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:10.824915 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal" Jul 12 18:25:10.873284 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:10.838741 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 18:25:10.873284 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:10.839956 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-170.ec2.internal" Jul 12 18:25:10.873284 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:10.849125 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 18:25:11.125827 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.064862 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 18:25:11.213241 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.201335 2585 apiserver.go:52] "Watching apiserver" Jul 12 18:25:11.214642 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.213290 2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 12 18:25:11.217654 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.215806 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-bx895","kube-system/global-pull-secret-syncer-j5bpc","kube-system/konnectivity-agent-cwkh5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq","openshift-dns/node-resolver-x7b84","openshift-image-registry/node-ca-5dzg6","openshift-multus/multus-9ph4f","openshift-multus/multus-additional-cni-plugins-llfzt","openshift-ovn-kubernetes/ovnkube-node-gswlt","kube-system/kube-apiserver-proxy-ip-10-0-140-170.ec2.internal","openshift-cluster-node-tuning-operator/tuned-5s6z5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal","openshift-multus/network-metrics-daemon-rgkpj","openshift-network-diagnostics/network-check-target-8q6bw"] Jul 12 18:25:11.220669 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.218556 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.222761 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.220964 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:11.222761 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:11.221047 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:11.222761 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.221488 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 12 18:25:11.222761 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.221535 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 12 18:25:11.222761 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.221493 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 12 18:25:11.222761 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.221910 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 12 18:25:11.222761 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.222046 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l24gz\"" Jul 12 18:25:11.225877 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.223660 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cwkh5" Jul 12 18:25:11.225877 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.225118 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 12 18:25:11.228868 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.226933 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 12 18:25:11.228868 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.226976 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 12 18:25:11.228868 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.227308 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jzw5f\"" Jul 12 18:25:11.228868 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.228042 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.230510 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x7b84" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.231151 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.231271 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.231477 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.231634 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gxlzj\"" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.232881 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.233180 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ddgp7\"" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.233507 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.233619 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5dzg6" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.235926 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.236518 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.236549 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8q5cp\"" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.236717 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.236717 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237218 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237575 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-multus-conf-dir\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237606 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a8d8139-c043-4f8c-b49e-76a54608b5af-agent-certs\") pod \"konnectivity-agent-cwkh5\" (UID: \"5a8d8139-c043-4f8c-b49e-76a54608b5af\") " pod="kube-system/konnectivity-agent-cwkh5" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237631 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-os-release\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237685 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-var-lib-cni-multus\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237712 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-var-lib-kubelet\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237737 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a8d8139-c043-4f8c-b49e-76a54608b5af-konnectivity-ca\") pod \"konnectivity-agent-cwkh5\" (UID: \"5a8d8139-c043-4f8c-b49e-76a54608b5af\") " pod="kube-system/konnectivity-agent-cwkh5" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237762 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-sys-fs\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237785 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-cnibin\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237811 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4122547-ea68-4da7-af41-996662132ed6-cni-binary-copy\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.288045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237834 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-etc-kubernetes\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237860 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs\") pod \"network-metrics-daemon-rgkpj\" (UID: \"87644bbc-d561-4ca5-8b2b-38bf70f01efb\") " pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237909 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-device-dir\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237933 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-multus-cni-dir\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237957 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-multus-socket-dir-parent\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.237982 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-socket-dir\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.238040 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-registration-dir\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.238066 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-run-k8s-cni-cncf-io\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.238093 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c4122547-ea68-4da7-af41-996662132ed6-multus-daemon-config\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.238122 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-etc-selinux\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.238159 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cbq2\" (UniqueName: \"kubernetes.io/projected/c4122547-ea68-4da7-af41-996662132ed6-kube-api-access-5cbq2\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.238184 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9kmt\" (UniqueName: \"kubernetes.io/projected/87644bbc-d561-4ca5-8b2b-38bf70f01efb-kube-api-access-c9kmt\") pod \"network-metrics-daemon-rgkpj\" (UID: \"87644bbc-d561-4ca5-8b2b-38bf70f01efb\") " pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.238213 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb4hj\" (UniqueName: \"kubernetes.io/projected/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-kube-api-access-mb4hj\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.238246 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-system-cni-dir\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.238287 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-run-netns\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.238325 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-hostroot\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.291281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.238376 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.238403 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-var-lib-cni-bin\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.238434 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-run-multus-certs\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.239456 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.239738 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8dzdf\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.240229 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.241231 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.241508 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.241648 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bx895" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.242192 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.242577 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.242746 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:11.242815 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.244440 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.244594 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.244888 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mtkd7\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.245063 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.246122 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:11.246196 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.249680 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.252447 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.253097 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.254683 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2dxcb\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.256385 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.259736 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nlglj\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.259771 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.259968 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.267890 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cp7bz" Jul 12 18:25:11.292236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.276448 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cp7bz" Jul 12 18:25:11.306388 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.305043 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal" event={"ID":"b1c54e9c324d378d37710ac8849ecf53","Type":"ContainerStarted","Data":"813b2f6a76b6d9c9ded60e3f248a372b05b94cf8b2230d5e92fe6de2a32bd867"} Jul 12 18:25:11.309927 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.306786 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-170.ec2.internal" event={"ID":"ec39bd46f4e845dad75a1dba09eb9d6f","Type":"ContainerStarted","Data":"766cdc02dc399aa7fe1cfdb9ab46d820f7376f6a9771332656b7eb24d076e0dc"} Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.327650 2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.338718 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd93dddb-eaaa-470d-84a4-17c4f61493e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.338758 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64c451f1-c731-40c5-a837-b515e65009e0-host\") pod \"node-ca-5dzg6\" (UID: \"64c451f1-c731-40c5-a837-b515e65009e0\") " pod="openshift-image-registry/node-ca-5dzg6" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.338786 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e0391cb3-5bc7-4fb5-9d22-f831b4125304-ovnkube-config\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.338812 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e0391cb3-5bc7-4fb5-9d22-f831b4125304-ovnkube-script-lib\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.338836 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-systemd\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.338858 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-run\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.338889 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-multus-cni-dir\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.338915 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-multus-socket-dir-parent\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.338939 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-systemd-units\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.338963 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-run-ovn\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.338988 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e0391cb3-5bc7-4fb5-9d22-f831b4125304-env-overrides\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339027 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-sysctl-conf\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339057 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-socket-dir\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339088 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-registration-dir\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339114 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-run-k8s-cni-cncf-io\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.369441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339145 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c4122547-ea68-4da7-af41-996662132ed6-multus-daemon-config\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339170 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsl97\" (UniqueName: \"kubernetes.io/projected/e0391cb3-5bc7-4fb5-9d22-f831b4125304-kube-api-access-vsl97\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339196 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-etc-selinux\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339231 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cbq2\" (UniqueName: \"kubernetes.io/projected/c4122547-ea68-4da7-af41-996662132ed6-kube-api-access-5cbq2\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339263 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-kubelet\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339286 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64c451f1-c731-40c5-a837-b515e65009e0-serviceca\") pod \"node-ca-5dzg6\" (UID: \"64c451f1-c731-40c5-a837-b515e65009e0\") " pod="openshift-image-registry/node-ca-5dzg6" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339311 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-var-lib-openvswitch\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339336 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-run-systemd\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339363 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339388 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w88sh\" (UniqueName: \"kubernetes.io/projected/fa50005b-ad03-474a-a648-f9fde6a932a6-kube-api-access-w88sh\") pod \"node-resolver-x7b84\" (UID: \"fa50005b-ad03-474a-a648-f9fde6a932a6\") " pod="openshift-dns/node-resolver-x7b84" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339414 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsbl9\" (UniqueName: \"kubernetes.io/projected/dd93dddb-eaaa-470d-84a4-17c4f61493e8-kube-api-access-dsbl9\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339441 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbqnf\" (UniqueName: \"kubernetes.io/projected/64c451f1-c731-40c5-a837-b515e65009e0-kube-api-access-wbqnf\") pod \"node-ca-5dzg6\" (UID: \"64c451f1-c731-40c5-a837-b515e65009e0\") " pod="openshift-image-registry/node-ca-5dzg6" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339582 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-registration-dir\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339620 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-socket-dir\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339640 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-run-k8s-cni-cncf-io\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339681 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-run-ovn-kubernetes\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.371137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339754 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e-iptables-alerter-script\") pod \"iptables-alerter-bx895\" (UID: \"aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e\") " pod="openshift-network-operator/iptables-alerter-bx895" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339781 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecd4d59c-8720-4481-ad1b-f4943dff29a4-tmp\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339812 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-multus-conf-dir\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339839 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a8d8139-c043-4f8c-b49e-76a54608b5af-agent-certs\") pod \"konnectivity-agent-cwkh5\" (UID: \"5a8d8139-c043-4f8c-b49e-76a54608b5af\") " pod="kube-system/konnectivity-agent-cwkh5" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339866 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd93dddb-eaaa-470d-84a4-17c4f61493e8-cnibin\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339893 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ad0a79f-69ee-49e8-90eb-c27482619b69-kubelet-config\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339919 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-os-release\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339946 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-var-lib-cni-multus\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.339992 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-var-lib-kubelet\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340023 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a8d8139-c043-4f8c-b49e-76a54608b5af-konnectivity-ca\") pod \"konnectivity-agent-cwkh5\" (UID: \"5a8d8139-c043-4f8c-b49e-76a54608b5af\") " pod="kube-system/konnectivity-agent-cwkh5" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340055 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-run-netns\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340084 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-kubernetes\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340121 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmkbd\" (UniqueName: \"kubernetes.io/projected/ecd4d59c-8720-4481-ad1b-f4943dff29a4-kube-api-access-rmkbd\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340157 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-sys-fs\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340182 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-cnibin\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340208 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs\") pod \"network-metrics-daemon-rgkpj\" (UID: \"87644bbc-d561-4ca5-8b2b-38bf70f01efb\") " pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340237 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa50005b-ad03-474a-a648-f9fde6a932a6-tmp-dir\") pod \"node-resolver-x7b84\" (UID: \"fa50005b-ad03-474a-a648-f9fde6a932a6\") " pod="openshift-dns/node-resolver-x7b84" Jul 12 18:25:11.372500 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340268 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340319 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-tuned\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340348 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-device-dir\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340322 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c4122547-ea68-4da7-af41-996662132ed6-multus-daemon-config\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340376 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd93dddb-eaaa-470d-84a4-17c4f61493e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340404 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-slash\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340430 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-etc-openvswitch\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340468 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-sysconfig\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340504 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd93dddb-eaaa-470d-84a4-17c4f61493e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340532 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-run-openvswitch\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340537 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-etc-selinux\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340559 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-cni-bin\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340584 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-sysctl-d\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340616 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9kmt\" (UniqueName: \"kubernetes.io/projected/87644bbc-d561-4ca5-8b2b-38bf70f01efb-kube-api-access-c9kmt\") pod \"network-metrics-daemon-rgkpj\" (UID: \"87644bbc-d561-4ca5-8b2b-38bf70f01efb\") " pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340651 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd93dddb-eaaa-470d-84a4-17c4f61493e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340680 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0391cb3-5bc7-4fb5-9d22-f831b4125304-ovn-node-metrics-cert\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.375057 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340721 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-sys\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340751 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb4hj\" (UniqueName: \"kubernetes.io/projected/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-kube-api-access-mb4hj\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340797 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-system-cni-dir\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.340848 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-run-netns\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341224 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341253 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-system-cni-dir\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341286 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-hostroot\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341300 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-multus-conf-dir\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341324 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e-host-slash\") pod \"iptables-alerter-bx895\" (UID: \"aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e\") " pod="openshift-network-operator/iptables-alerter-bx895" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341355 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ns4m\" (UniqueName: \"kubernetes.io/projected/aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e-kube-api-access-5ns4m\") pod \"iptables-alerter-bx895\" (UID: \"aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e\") " pod="openshift-network-operator/iptables-alerter-bx895" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341394 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-lib-modules\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341455 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-host\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341484 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-var-lib-cni-bin\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341515 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-run-multus-certs\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341544 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz8h8\" (UniqueName: \"kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8\") pod \"network-check-target-8q6bw\" (UID: \"7abc23a1-1bde-4046-a60d-cf3e93e64f63\") " pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341571 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd93dddb-eaaa-470d-84a4-17c4f61493e8-os-release\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341601 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-var-lib-kubelet\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.376267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341630 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa50005b-ad03-474a-a648-f9fde6a932a6-hosts-file\") pod \"node-resolver-x7b84\" (UID: \"fa50005b-ad03-474a-a648-f9fde6a932a6\") " pod="openshift-dns/node-resolver-x7b84" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341658 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd93dddb-eaaa-470d-84a4-17c4f61493e8-system-cni-dir\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341730 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-node-log\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341767 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-log-socket\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341780 2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341804 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-cni-netd\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341831 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341891 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-modprobe-d\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341923 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ad0a79f-69ee-49e8-90eb-c27482619b69-dbus\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341953 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4122547-ea68-4da7-af41-996662132ed6-cni-binary-copy\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.341981 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-etc-kubernetes\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.342090 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-etc-kubernetes\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.342146 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-run-netns\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.342348 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-multus-cni-dir\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.342405 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-device-dir\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.342475 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-os-release\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.342519 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-var-lib-cni-multus\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.342558 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-var-lib-kubelet\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.379545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.343047 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-cnibin\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.380704 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.343157 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a8d8139-c043-4f8c-b49e-76a54608b5af-konnectivity-ca\") pod \"konnectivity-agent-cwkh5\" (UID: \"5a8d8139-c043-4f8c-b49e-76a54608b5af\") " pod="kube-system/konnectivity-agent-cwkh5" Jul 12 18:25:11.380704 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:11.343276 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:25:11.380704 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:11.343410 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs podName:87644bbc-d561-4ca5-8b2b-38bf70f01efb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:11.843381583 +0000 UTC m=+3.042848611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs") pod "network-metrics-daemon-rgkpj" (UID: "87644bbc-d561-4ca5-8b2b-38bf70f01efb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:25:11.380704 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.343589 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-multus-socket-dir-parent\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.380704 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.343618 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-var-lib-cni-bin\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.380704 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.343653 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-sys-fs\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.380704 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.343702 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-host-run-multus-certs\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.380704 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.343726 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c4122547-ea68-4da7-af41-996662132ed6-hostroot\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.380704 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.344253 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4122547-ea68-4da7-af41-996662132ed6-cni-binary-copy\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.380704 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.346761 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a8d8139-c043-4f8c-b49e-76a54608b5af-agent-certs\") pod \"konnectivity-agent-cwkh5\" (UID: \"5a8d8139-c043-4f8c-b49e-76a54608b5af\") " pod="kube-system/konnectivity-agent-cwkh5" Jul 12 18:25:11.380704 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.351983 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cbq2\" (UniqueName: \"kubernetes.io/projected/c4122547-ea68-4da7-af41-996662132ed6-kube-api-access-5cbq2\") pod \"multus-9ph4f\" (UID: \"c4122547-ea68-4da7-af41-996662132ed6\") " pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.391159 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb4hj\" (UniqueName: \"kubernetes.io/projected/a62e223b-ddd1-4dd2-92e6-c87f9b606ece-kube-api-access-mb4hj\") pod \"aws-ebs-csi-driver-node-s47kq\" (UID: \"a62e223b-ddd1-4dd2-92e6-c87f9b606ece\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.393077 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9kmt\" (UniqueName: \"kubernetes.io/projected/87644bbc-d561-4ca5-8b2b-38bf70f01efb-kube-api-access-c9kmt\") pod \"network-metrics-daemon-rgkpj\" (UID: \"87644bbc-d561-4ca5-8b2b-38bf70f01efb\") " pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443309 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsbl9\" (UniqueName: \"kubernetes.io/projected/dd93dddb-eaaa-470d-84a4-17c4f61493e8-kube-api-access-dsbl9\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443359 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbqnf\" (UniqueName: \"kubernetes.io/projected/64c451f1-c731-40c5-a837-b515e65009e0-kube-api-access-wbqnf\") pod \"node-ca-5dzg6\" (UID: \"64c451f1-c731-40c5-a837-b515e65009e0\") " pod="openshift-image-registry/node-ca-5dzg6" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443392 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-run-ovn-kubernetes\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443422 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e-iptables-alerter-script\") pod \"iptables-alerter-bx895\" (UID: \"aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e\") " pod="openshift-network-operator/iptables-alerter-bx895" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443453 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecd4d59c-8720-4481-ad1b-f4943dff29a4-tmp\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443482 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd93dddb-eaaa-470d-84a4-17c4f61493e8-cnibin\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443509 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ad0a79f-69ee-49e8-90eb-c27482619b69-kubelet-config\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443538 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-run-netns\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443568 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-kubernetes\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443593 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmkbd\" (UniqueName: \"kubernetes.io/projected/ecd4d59c-8720-4481-ad1b-f4943dff29a4-kube-api-access-rmkbd\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443635 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa50005b-ad03-474a-a648-f9fde6a932a6-tmp-dir\") pod \"node-resolver-x7b84\" (UID: \"fa50005b-ad03-474a-a648-f9fde6a932a6\") " pod="openshift-dns/node-resolver-x7b84" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443677 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443701 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-tuned\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443736 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd93dddb-eaaa-470d-84a4-17c4f61493e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.460070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443763 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-slash\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443789 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-etc-openvswitch\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443816 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-sysconfig\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443844 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd93dddb-eaaa-470d-84a4-17c4f61493e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443868 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-run-openvswitch\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443892 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-cni-bin\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443917 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-sysctl-d\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443953 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd93dddb-eaaa-470d-84a4-17c4f61493e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.443980 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0391cb3-5bc7-4fb5-9d22-f831b4125304-ovn-node-metrics-cert\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444035 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-sys\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444068 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e-host-slash\") pod \"iptables-alerter-bx895\" (UID: \"aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e\") " pod="openshift-network-operator/iptables-alerter-bx895" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444093 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ns4m\" (UniqueName: \"kubernetes.io/projected/aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e-kube-api-access-5ns4m\") pod \"iptables-alerter-bx895\" (UID: \"aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e\") " pod="openshift-network-operator/iptables-alerter-bx895" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444117 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-lib-modules\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444140 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-host\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444168 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz8h8\" (UniqueName: \"kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8\") pod \"network-check-target-8q6bw\" (UID: \"7abc23a1-1bde-4046-a60d-cf3e93e64f63\") " pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444194 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd93dddb-eaaa-470d-84a4-17c4f61493e8-os-release\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444218 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-var-lib-kubelet\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.466302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444245 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa50005b-ad03-474a-a648-f9fde6a932a6-hosts-file\") pod \"node-resolver-x7b84\" (UID: \"fa50005b-ad03-474a-a648-f9fde6a932a6\") " pod="openshift-dns/node-resolver-x7b84" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444270 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd93dddb-eaaa-470d-84a4-17c4f61493e8-system-cni-dir\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444302 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-node-log\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444331 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-log-socket\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444357 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-cni-netd\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444387 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444414 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-modprobe-d\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444441 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ad0a79f-69ee-49e8-90eb-c27482619b69-dbus\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444471 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd93dddb-eaaa-470d-84a4-17c4f61493e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444497 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64c451f1-c731-40c5-a837-b515e65009e0-host\") pod \"node-ca-5dzg6\" (UID: \"64c451f1-c731-40c5-a837-b515e65009e0\") " pod="openshift-image-registry/node-ca-5dzg6" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444521 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e0391cb3-5bc7-4fb5-9d22-f831b4125304-ovnkube-config\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444546 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e0391cb3-5bc7-4fb5-9d22-f831b4125304-ovnkube-script-lib\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444575 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-systemd\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444601 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-run\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444629 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-systemd-units\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444654 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-run-ovn\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444680 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e0391cb3-5bc7-4fb5-9d22-f831b4125304-env-overrides\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.467380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444707 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-sysctl-conf\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444742 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsl97\" (UniqueName: \"kubernetes.io/projected/e0391cb3-5bc7-4fb5-9d22-f831b4125304-kube-api-access-vsl97\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444772 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-kubelet\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444797 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64c451f1-c731-40c5-a837-b515e65009e0-serviceca\") pod \"node-ca-5dzg6\" (UID: \"64c451f1-c731-40c5-a837-b515e65009e0\") " pod="openshift-image-registry/node-ca-5dzg6" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444826 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-var-lib-openvswitch\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444854 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-run-systemd\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.444883 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w88sh\" (UniqueName: \"kubernetes.io/projected/fa50005b-ad03-474a-a648-f9fde6a932a6-kube-api-access-w88sh\") pod \"node-resolver-x7b84\" (UID: \"fa50005b-ad03-474a-a648-f9fde6a932a6\") " pod="openshift-dns/node-resolver-x7b84" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.445642 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-run-ovn-kubernetes\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.446377 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e-iptables-alerter-script\") pod \"iptables-alerter-bx895\" (UID: \"aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e\") " pod="openshift-network-operator/iptables-alerter-bx895" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.446852 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd93dddb-eaaa-470d-84a4-17c4f61493e8-os-release\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.446918 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd93dddb-eaaa-470d-84a4-17c4f61493e8-cnibin\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.446972 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ad0a79f-69ee-49e8-90eb-c27482619b69-kubelet-config\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.447044 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-run-netns\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.447111 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-kubernetes\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.447626 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa50005b-ad03-474a-a648-f9fde6a932a6-tmp-dir\") pod \"node-resolver-x7b84\" (UID: \"fa50005b-ad03-474a-a648-f9fde6a932a6\") " pod="openshift-dns/node-resolver-x7b84" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.447731 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.449856 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-sysctl-d\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.468338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450035 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd93dddb-eaaa-470d-84a4-17c4f61493e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450052 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-sysconfig\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450130 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-slash\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450181 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-etc-openvswitch\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450230 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-run-openvswitch\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450290 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-cni-netd\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450362 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-var-lib-kubelet\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450403 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa50005b-ad03-474a-a648-f9fde6a932a6-hosts-file\") pod \"node-resolver-x7b84\" (UID: \"fa50005b-ad03-474a-a648-f9fde6a932a6\") " pod="openshift-dns/node-resolver-x7b84" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450439 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd93dddb-eaaa-470d-84a4-17c4f61493e8-system-cni-dir\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450477 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-node-log\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450503 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-log-socket\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450530 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-cni-bin\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450570 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ad0a79f-69ee-49e8-90eb-c27482619b69-dbus\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:11.450635 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450669 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd93dddb-eaaa-470d-84a4-17c4f61493e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:11.450695 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret podName:6ad0a79f-69ee-49e8-90eb-c27482619b69 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:11.950676165 +0000 UTC m=+3.150143181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret") pod "global-pull-secret-syncer-j5bpc" (UID: "6ad0a79f-69ee-49e8-90eb-c27482619b69") : object "kube-system"/"original-pull-secret" not registered Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450739 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e0391cb3-5bc7-4fb5-9d22-f831b4125304-ovnkube-config\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.469323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450749 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-host-kubelet\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450777 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-modprobe-d\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450857 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64c451f1-c731-40c5-a837-b515e65009e0-host\") pod \"node-ca-5dzg6\" (UID: \"64c451f1-c731-40c5-a837-b515e65009e0\") " pod="openshift-image-registry/node-ca-5dzg6" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.450965 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e-host-slash\") pod \"iptables-alerter-bx895\" (UID: \"aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e\") " pod="openshift-network-operator/iptables-alerter-bx895" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.451073 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-sysctl-conf\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.451485 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd93dddb-eaaa-470d-84a4-17c4f61493e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.451564 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-sys\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.453556 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecd4d59c-8720-4481-ad1b-f4943dff29a4-tmp\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.453630 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd93dddb-eaaa-470d-84a4-17c4f61493e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.453692 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e0391cb3-5bc7-4fb5-9d22-f831b4125304-env-overrides\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.453800 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-var-lib-openvswitch\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.453925 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0391cb3-5bc7-4fb5-9d22-f831b4125304-ovn-node-metrics-cert\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.454041 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-systemd-units\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.454229 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-run-systemd\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.454267 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64c451f1-c731-40c5-a837-b515e65009e0-serviceca\") pod \"node-ca-5dzg6\" (UID: \"64c451f1-c731-40c5-a837-b515e65009e0\") " pod="openshift-image-registry/node-ca-5dzg6" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.454344 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0391cb3-5bc7-4fb5-9d22-f831b4125304-run-ovn\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.454390 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-lib-modules\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.454413 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-host\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.470197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.454478 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-systemd\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.471311 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.454556 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e0391cb3-5bc7-4fb5-9d22-f831b4125304-ovnkube-script-lib\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.471311 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.454647 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ecd4d59c-8720-4481-ad1b-f4943dff29a4-run\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.471311 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.455180 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ecd4d59c-8720-4481-ad1b-f4943dff29a4-etc-tuned\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.471311 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:11.460239 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 18:25:11.471311 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:11.460260 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 18:25:11.471311 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:11.460275 2585 projected.go:196] Error preparing data for projected volume kube-api-access-wz8h8 for pod openshift-network-diagnostics/network-check-target-8q6bw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:25:11.471311 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:11.460350 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8 podName:7abc23a1-1bde-4046-a60d-cf3e93e64f63 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:11.96033021 +0000 UTC m=+3.159797237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wz8h8" (UniqueName: "kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8") pod "network-check-target-8q6bw" (UID: "7abc23a1-1bde-4046-a60d-cf3e93e64f63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:25:11.471311 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.465070 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w88sh\" (UniqueName: \"kubernetes.io/projected/fa50005b-ad03-474a-a648-f9fde6a932a6-kube-api-access-w88sh\") pod \"node-resolver-x7b84\" (UID: \"fa50005b-ad03-474a-a648-f9fde6a932a6\") " pod="openshift-dns/node-resolver-x7b84" Jul 12 18:25:11.471311 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.466954 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsbl9\" (UniqueName: \"kubernetes.io/projected/dd93dddb-eaaa-470d-84a4-17c4f61493e8-kube-api-access-dsbl9\") pod \"multus-additional-cni-plugins-llfzt\" (UID: \"dd93dddb-eaaa-470d-84a4-17c4f61493e8\") " pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.471311 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.467260 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbqnf\" (UniqueName: \"kubernetes.io/projected/64c451f1-c731-40c5-a837-b515e65009e0-kube-api-access-wbqnf\") pod \"node-ca-5dzg6\" (UID: \"64c451f1-c731-40c5-a837-b515e65009e0\") " pod="openshift-image-registry/node-ca-5dzg6" Jul 12 18:25:11.471311 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.468291 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsl97\" (UniqueName: \"kubernetes.io/projected/e0391cb3-5bc7-4fb5-9d22-f831b4125304-kube-api-access-vsl97\") pod \"ovnkube-node-gswlt\" (UID: \"e0391cb3-5bc7-4fb5-9d22-f831b4125304\") " pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.471311 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.468767 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmkbd\" (UniqueName: \"kubernetes.io/projected/ecd4d59c-8720-4481-ad1b-f4943dff29a4-kube-api-access-rmkbd\") pod \"tuned-5s6z5\" (UID: \"ecd4d59c-8720-4481-ad1b-f4943dff29a4\") " pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.471311 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.469813 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ns4m\" (UniqueName: \"kubernetes.io/projected/aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e-kube-api-access-5ns4m\") pod \"iptables-alerter-bx895\" (UID: \"aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e\") " pod="openshift-network-operator/iptables-alerter-bx895" Jul 12 18:25:11.534798 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.534758 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9ph4f" Jul 12 18:25:11.543934 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.543905 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cwkh5" Jul 12 18:25:11.549703 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:11.549663 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4122547_ea68_4da7_af41_996662132ed6.slice/crio-785a292016ce02cca0a15f8c9f9c71be9b51801c5ae2b0b84dfbfb961f923f14 WatchSource:0}: Error finding container 785a292016ce02cca0a15f8c9f9c71be9b51801c5ae2b0b84dfbfb961f923f14: Status 404 returned error can't find the container with id 785a292016ce02cca0a15f8c9f9c71be9b51801c5ae2b0b84dfbfb961f923f14 Jul 12 18:25:11.558492 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:11.558457 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a8d8139_c043_4f8c_b49e_76a54608b5af.slice/crio-a2f1d61964931b1f001f913631685a29361bbf7091816c65d6219536620171fe WatchSource:0}: Error finding container a2f1d61964931b1f001f913631685a29361bbf7091816c65d6219536620171fe: Status 404 returned error can't find the container with id a2f1d61964931b1f001f913631685a29361bbf7091816c65d6219536620171fe Jul 12 18:25:11.559650 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.559624 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" Jul 12 18:25:11.569932 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.569888 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x7b84" Jul 12 18:25:11.577170 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.577140 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5dzg6" Jul 12 18:25:11.590532 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.590493 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:11.691172 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.691078 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bx895" Jul 12 18:25:11.713789 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.713754 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" Jul 12 18:25:11.721027 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.720998 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-llfzt" Jul 12 18:25:11.847016 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:11.846965 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs\") pod \"network-metrics-daemon-rgkpj\" (UID: \"87644bbc-d561-4ca5-8b2b-38bf70f01efb\") " pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:11.847216 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:11.847150 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:25:11.847276 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:11.847237 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs podName:87644bbc-d561-4ca5-8b2b-38bf70f01efb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:12.847213691 +0000 UTC m=+4.046680704 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs") pod "network-metrics-daemon-rgkpj" (UID: "87644bbc-d561-4ca5-8b2b-38bf70f01efb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:25:12.048808 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:12.048694 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz8h8\" (UniqueName: \"kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8\") pod \"network-check-target-8q6bw\" (UID: \"7abc23a1-1bde-4046-a60d-cf3e93e64f63\") " pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:12.048808 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:12.048753 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:12.049065 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:12.048886 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 18:25:12.049065 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:12.048895 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 18:25:12.049065 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:12.048915 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 18:25:12.049065 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:12.048928 2585 projected.go:196] Error preparing data for projected volume kube-api-access-wz8h8 for pod openshift-network-diagnostics/network-check-target-8q6bw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:25:12.049065 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:12.048951 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret podName:6ad0a79f-69ee-49e8-90eb-c27482619b69 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:13.048931614 +0000 UTC m=+4.248398655 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret") pod "global-pull-secret-syncer-j5bpc" (UID: "6ad0a79f-69ee-49e8-90eb-c27482619b69") : object "kube-system"/"original-pull-secret" not registered Jul 12 18:25:12.049065 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:12.048981 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8 podName:7abc23a1-1bde-4046-a60d-cf3e93e64f63 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:13.048969591 +0000 UTC m=+4.248436623 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wz8h8" (UniqueName: "kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8") pod "network-check-target-8q6bw" (UID: "7abc23a1-1bde-4046-a60d-cf3e93e64f63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:25:12.279695 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:12.279656 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 18:20:11 +0000 UTC" deadline="2028-04-09 15:46:32.198413474 +0000 UTC" Jul 12 18:25:12.279695 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:12.279690 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15285h21m19.918727315s" Jul 12 18:25:12.310115 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:12.309966 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cwkh5" event={"ID":"5a8d8139-c043-4f8c-b49e-76a54608b5af","Type":"ContainerStarted","Data":"a2f1d61964931b1f001f913631685a29361bbf7091816c65d6219536620171fe"} Jul 12 18:25:12.311192 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:12.311164 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9ph4f" event={"ID":"c4122547-ea68-4da7-af41-996662132ed6","Type":"ContainerStarted","Data":"785a292016ce02cca0a15f8c9f9c71be9b51801c5ae2b0b84dfbfb961f923f14"} Jul 12 18:25:12.711298 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:12.711272 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda62e223b_ddd1_4dd2_92e6_c87f9b606ece.slice/crio-237a7f3aa7f85080b44180a4860dd7b3e1bce0b550b29a813e17cfc71b2e69bd WatchSource:0}: Error finding container 237a7f3aa7f85080b44180a4860dd7b3e1bce0b550b29a813e17cfc71b2e69bd: Status 404 returned error can't find the container with id 237a7f3aa7f85080b44180a4860dd7b3e1bce0b550b29a813e17cfc71b2e69bd Jul 12 18:25:12.728914 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:12.728877 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae3b7fb_6dd9_4a5f_9ada_88a9b9e3663e.slice/crio-61a969db1d9ea19d347ad2d29e6054f6f3706447b6e2409e6bcf44a2a21eee30 WatchSource:0}: Error finding container 61a969db1d9ea19d347ad2d29e6054f6f3706447b6e2409e6bcf44a2a21eee30: Status 404 returned error can't find the container with id 61a969db1d9ea19d347ad2d29e6054f6f3706447b6e2409e6bcf44a2a21eee30 Jul 12 18:25:12.731573 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:12.731461 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecd4d59c_8720_4481_ad1b_f4943dff29a4.slice/crio-a5dd2c3f5757eb76c911052e3cde84f415d017b93425fb3f78dc7788f330adcc WatchSource:0}: Error finding container a5dd2c3f5757eb76c911052e3cde84f415d017b93425fb3f78dc7788f330adcc: Status 404 returned error can't find the container with id a5dd2c3f5757eb76c911052e3cde84f415d017b93425fb3f78dc7788f330adcc Jul 12 18:25:12.732442 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:12.732413 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd93dddb_eaaa_470d_84a4_17c4f61493e8.slice/crio-4b2a232ccc869c8179b0d3fe31dbb7a4def5ea4ac75f18a9c169f30d0dfcd7bb WatchSource:0}: Error finding container 4b2a232ccc869c8179b0d3fe31dbb7a4def5ea4ac75f18a9c169f30d0dfcd7bb: Status 404 returned error can't find the container with id 4b2a232ccc869c8179b0d3fe31dbb7a4def5ea4ac75f18a9c169f30d0dfcd7bb Jul 12 18:25:12.733280 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:12.733248 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c451f1_c731_40c5_a837_b515e65009e0.slice/crio-f7a2af5467bcbe5232ea5ee593ffcc67632d0362634951a71ad3c2a45cd1e12d WatchSource:0}: Error finding container f7a2af5467bcbe5232ea5ee593ffcc67632d0362634951a71ad3c2a45cd1e12d: Status 404 returned error can't find the container with id f7a2af5467bcbe5232ea5ee593ffcc67632d0362634951a71ad3c2a45cd1e12d Jul 12 18:25:12.735945 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:12.735922 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0391cb3_5bc7_4fb5_9d22_f831b4125304.slice/crio-8da28597d6ba6d4025f559a27403896b33ec08d6dce1c1f8ac230a575c6dfc53 WatchSource:0}: Error finding container 8da28597d6ba6d4025f559a27403896b33ec08d6dce1c1f8ac230a575c6dfc53: Status 404 returned error can't find the container with id 8da28597d6ba6d4025f559a27403896b33ec08d6dce1c1f8ac230a575c6dfc53 Jul 12 18:25:12.736629 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:12.736483 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa50005b_ad03_474a_a648_f9fde6a932a6.slice/crio-e3d36c8cea2eb0bc87522d49965e2825663323acc38c6ffacb073cd29cfb7a66 WatchSource:0}: Error finding container e3d36c8cea2eb0bc87522d49965e2825663323acc38c6ffacb073cd29cfb7a66: Status 404 returned error can't find the container with id e3d36c8cea2eb0bc87522d49965e2825663323acc38c6ffacb073cd29cfb7a66 Jul 12 18:25:12.855116 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:12.855046 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs\") pod \"network-metrics-daemon-rgkpj\" (UID: \"87644bbc-d561-4ca5-8b2b-38bf70f01efb\") " pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:12.855282 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:12.855194 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:25:12.855282 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:12.855277 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs podName:87644bbc-d561-4ca5-8b2b-38bf70f01efb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:14.85525791 +0000 UTC m=+6.054724923 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs") pod "network-metrics-daemon-rgkpj" (UID: "87644bbc-d561-4ca5-8b2b-38bf70f01efb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:25:13.056431 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.056370 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz8h8\" (UniqueName: \"kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8\") pod \"network-check-target-8q6bw\" (UID: \"7abc23a1-1bde-4046-a60d-cf3e93e64f63\") " pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:13.056617 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.056439 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:13.056617 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:13.056607 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 18:25:13.056721 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:13.056679 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret podName:6ad0a79f-69ee-49e8-90eb-c27482619b69 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:15.056657663 +0000 UTC m=+6.256124692 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret") pod "global-pull-secret-syncer-j5bpc" (UID: "6ad0a79f-69ee-49e8-90eb-c27482619b69") : object "kube-system"/"original-pull-secret" not registered Jul 12 18:25:13.056918 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:13.056891 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 18:25:13.056918 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:13.056919 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 18:25:13.056918 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:13.056934 2585 projected.go:196] Error preparing data for projected volume kube-api-access-wz8h8 for pod openshift-network-diagnostics/network-check-target-8q6bw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:25:13.057164 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:13.057019 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8 podName:7abc23a1-1bde-4046-a60d-cf3e93e64f63 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:15.056999738 +0000 UTC m=+6.256466763 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wz8h8" (UniqueName: "kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8") pod "network-check-target-8q6bw" (UID: "7abc23a1-1bde-4046-a60d-cf3e93e64f63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:25:13.280320 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.280172 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 18:20:11 +0000 UTC" deadline="2028-01-18 19:00:36.893631561 +0000 UTC" Jul 12 18:25:13.280320 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.280214 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13320h35m23.613422953s" Jul 12 18:25:13.300866 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.300180 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:13.300866 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:13.300331 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:13.300866 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.300583 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:13.300866 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:13.300715 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:13.300866 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.300833 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:13.301277 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:13.300928 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:13.319248 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.319210 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" event={"ID":"e0391cb3-5bc7-4fb5-9d22-f831b4125304","Type":"ContainerStarted","Data":"8da28597d6ba6d4025f559a27403896b33ec08d6dce1c1f8ac230a575c6dfc53"} Jul 12 18:25:13.327416 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.327372 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5dzg6" event={"ID":"64c451f1-c731-40c5-a837-b515e65009e0","Type":"ContainerStarted","Data":"f7a2af5467bcbe5232ea5ee593ffcc67632d0362634951a71ad3c2a45cd1e12d"} Jul 12 18:25:13.329873 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.329829 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bx895" event={"ID":"aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e","Type":"ContainerStarted","Data":"61a969db1d9ea19d347ad2d29e6054f6f3706447b6e2409e6bcf44a2a21eee30"} Jul 12 18:25:13.332308 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.332277 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" event={"ID":"a62e223b-ddd1-4dd2-92e6-c87f9b606ece","Type":"ContainerStarted","Data":"237a7f3aa7f85080b44180a4860dd7b3e1bce0b550b29a813e17cfc71b2e69bd"} Jul 12 18:25:13.340077 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.338893 2585 generic.go:356] "Generic (PLEG): container finished" podID="b1c54e9c324d378d37710ac8849ecf53" containerID="1664b588ccbac63ea8bc53ec7d1431ed981b072148fff12da63017041e9fb0fb" exitCode=0 Jul 12 18:25:13.340077 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.338988 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal" event={"ID":"b1c54e9c324d378d37710ac8849ecf53","Type":"ContainerDied","Data":"1664b588ccbac63ea8bc53ec7d1431ed981b072148fff12da63017041e9fb0fb"} Jul 12 18:25:13.349085 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.349050 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x7b84" event={"ID":"fa50005b-ad03-474a-a648-f9fde6a932a6","Type":"ContainerStarted","Data":"e3d36c8cea2eb0bc87522d49965e2825663323acc38c6ffacb073cd29cfb7a66"} Jul 12 18:25:13.360545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.358880 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llfzt" event={"ID":"dd93dddb-eaaa-470d-84a4-17c4f61493e8","Type":"ContainerStarted","Data":"4b2a232ccc869c8179b0d3fe31dbb7a4def5ea4ac75f18a9c169f30d0dfcd7bb"} Jul 12 18:25:13.363751 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.363714 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" event={"ID":"ecd4d59c-8720-4481-ad1b-f4943dff29a4","Type":"ContainerStarted","Data":"a5dd2c3f5757eb76c911052e3cde84f415d017b93425fb3f78dc7788f330adcc"} Jul 12 18:25:13.367959 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.367895 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-170.ec2.internal" event={"ID":"ec39bd46f4e845dad75a1dba09eb9d6f","Type":"ContainerStarted","Data":"2b913045267aeff150874a3b33ed8fd98638ed2404ff14e94bbdce4d9c5d340f"} Jul 12 18:25:13.390675 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:13.390584 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-170.ec2.internal" podStartSLOduration=2.390555863 podStartE2EDuration="2.390555863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:13.390442643 +0000 UTC m=+4.589909759" watchObservedRunningTime="2026-07-12 18:25:13.390555863 +0000 UTC m=+4.590022888" Jul 12 18:25:14.379460 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:14.379354 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal" event={"ID":"b1c54e9c324d378d37710ac8849ecf53","Type":"ContainerStarted","Data":"4f8b5087815b142ce2521691b7caff65179910454bbf3dd72f7d12d549d5ec00"} Jul 12 18:25:14.870501 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:14.870394 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs\") pod \"network-metrics-daemon-rgkpj\" (UID: \"87644bbc-d561-4ca5-8b2b-38bf70f01efb\") " pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:14.870747 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:14.870564 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:25:14.870747 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:14.870635 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs podName:87644bbc-d561-4ca5-8b2b-38bf70f01efb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:18.870613062 +0000 UTC m=+10.070080087 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs") pod "network-metrics-daemon-rgkpj" (UID: "87644bbc-d561-4ca5-8b2b-38bf70f01efb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:25:15.072423 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:15.072379 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz8h8\" (UniqueName: \"kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8\") pod \"network-check-target-8q6bw\" (UID: \"7abc23a1-1bde-4046-a60d-cf3e93e64f63\") " pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:15.072632 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:15.072444 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:15.072704 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:15.072679 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 18:25:15.072809 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:15.072763 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret podName:6ad0a79f-69ee-49e8-90eb-c27482619b69 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:19.072741006 +0000 UTC m=+10.272208030 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret") pod "global-pull-secret-syncer-j5bpc" (UID: "6ad0a79f-69ee-49e8-90eb-c27482619b69") : object "kube-system"/"original-pull-secret" not registered Jul 12 18:25:15.072885 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:15.072876 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 18:25:15.072941 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:15.072892 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 18:25:15.072941 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:15.072905 2585 projected.go:196] Error preparing data for projected volume kube-api-access-wz8h8 for pod openshift-network-diagnostics/network-check-target-8q6bw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:25:15.073077 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:15.072945 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8 podName:7abc23a1-1bde-4046-a60d-cf3e93e64f63 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:19.072932708 +0000 UTC m=+10.272399724 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wz8h8" (UniqueName: "kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8") pod "network-check-target-8q6bw" (UID: "7abc23a1-1bde-4046-a60d-cf3e93e64f63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:25:15.299169 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:15.299079 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:15.299169 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:15.299127 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:15.299484 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:15.299090 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:15.299484 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:15.299223 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:15.299484 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:15.299306 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:15.299484 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:15.299387 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:17.298846 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:17.298811 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:17.299407 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:17.298993 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:17.299407 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:17.299399 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:17.299526 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:17.299501 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:17.299581 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:17.299555 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:17.299678 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:17.299642 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:18.908175 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:18.908129 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs\") pod \"network-metrics-daemon-rgkpj\" (UID: \"87644bbc-d561-4ca5-8b2b-38bf70f01efb\") " pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:18.908622 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:18.908259 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:25:18.908622 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:18.908345 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs podName:87644bbc-d561-4ca5-8b2b-38bf70f01efb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:26.908323188 +0000 UTC m=+18.107790210 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs") pod "network-metrics-daemon-rgkpj" (UID: "87644bbc-d561-4ca5-8b2b-38bf70f01efb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:25:19.110092 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:19.109484 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz8h8\" (UniqueName: \"kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8\") pod \"network-check-target-8q6bw\" (UID: \"7abc23a1-1bde-4046-a60d-cf3e93e64f63\") " pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:19.110092 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:19.109546 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:19.110092 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:19.109697 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 18:25:19.110092 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:19.109764 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret podName:6ad0a79f-69ee-49e8-90eb-c27482619b69 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:27.109746316 +0000 UTC m=+18.309213352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret") pod "global-pull-secret-syncer-j5bpc" (UID: "6ad0a79f-69ee-49e8-90eb-c27482619b69") : object "kube-system"/"original-pull-secret" not registered Jul 12 18:25:19.110092 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:19.109861 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 18:25:19.110092 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:19.109877 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 18:25:19.110092 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:19.109891 2585 projected.go:196] Error preparing data for projected volume kube-api-access-wz8h8 for pod openshift-network-diagnostics/network-check-target-8q6bw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:25:19.110092 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:19.109951 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8 podName:7abc23a1-1bde-4046-a60d-cf3e93e64f63 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:27.109921937 +0000 UTC m=+18.309388968 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wz8h8" (UniqueName: "kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8") pod "network-check-target-8q6bw" (UID: "7abc23a1-1bde-4046-a60d-cf3e93e64f63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:25:19.300440 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:19.299640 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:19.300440 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:19.299678 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:19.300440 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:19.299754 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:19.300440 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:19.300183 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:19.300440 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:19.300248 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:19.300440 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:19.300333 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:21.298252 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:21.298151 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:21.298681 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:21.298159 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:21.298681 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:21.298285 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:21.298681 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:21.298159 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:21.298681 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:21.298383 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:21.298681 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:21.298495 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:23.299127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:23.299095 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:23.299573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:23.299095 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:23.299573 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:23.299207 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:23.299573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:23.299095 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:23.299573 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:23.299358 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:23.299573 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:23.299374 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:25.298768 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:25.298730 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:25.299371 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:25.298730 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:25.299371 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:25.298875 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:25.299371 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:25.298730 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:25.299371 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:25.298968 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:25.299371 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:25.299042 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:26.970425 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:26.970383 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs\") pod \"network-metrics-daemon-rgkpj\" (UID: \"87644bbc-d561-4ca5-8b2b-38bf70f01efb\") " pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:26.971012 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:26.970523 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:25:26.971012 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:26.970604 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs podName:87644bbc-d561-4ca5-8b2b-38bf70f01efb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:42.970583053 +0000 UTC m=+34.170050064 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs") pod "network-metrics-daemon-rgkpj" (UID: "87644bbc-d561-4ca5-8b2b-38bf70f01efb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:25:27.172369 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:27.172328 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz8h8\" (UniqueName: \"kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8\") pod \"network-check-target-8q6bw\" (UID: \"7abc23a1-1bde-4046-a60d-cf3e93e64f63\") " pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:27.172369 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:27.172371 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:27.172618 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:27.172501 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 18:25:27.172618 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:27.172532 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 18:25:27.172618 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:27.172554 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 18:25:27.172618 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:27.172565 2585 projected.go:196] Error preparing data for projected volume kube-api-access-wz8h8 for pod openshift-network-diagnostics/network-check-target-8q6bw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:25:27.172618 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:27.172573 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret podName:6ad0a79f-69ee-49e8-90eb-c27482619b69 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:43.172554738 +0000 UTC m=+34.372021787 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret") pod "global-pull-secret-syncer-j5bpc" (UID: "6ad0a79f-69ee-49e8-90eb-c27482619b69") : object "kube-system"/"original-pull-secret" not registered Jul 12 18:25:27.172618 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:27.172609 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8 podName:7abc23a1-1bde-4046-a60d-cf3e93e64f63 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:43.172598164 +0000 UTC m=+34.372065172 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wz8h8" (UniqueName: "kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8") pod "network-check-target-8q6bw" (UID: "7abc23a1-1bde-4046-a60d-cf3e93e64f63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:25:27.298226 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:27.298192 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:27.298430 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:27.298241 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:27.298430 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:27.298202 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:27.298430 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:27.298319 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:27.298590 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:27.298444 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:27.298590 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:27.298531 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:29.298839 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:29.298809 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:29.299431 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:29.298917 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:29.299431 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:29.298960 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:29.299431 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:29.298962 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:29.299431 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:29.299046 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:29.299431 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:29.299146 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:31.298473 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:31.298435 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:31.298902 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:31.298436 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:31.298902 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:31.298562 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:31.298902 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:31.298443 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:31.298902 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:31.298658 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:31.298902 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:31.298710 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:32.431597 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.431192 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9ph4f" event={"ID":"c4122547-ea68-4da7-af41-996662132ed6","Type":"ContainerStarted","Data":"a747d4fc35177c44006d1a03d028d009928a8196aca13dd2e91f0fcc694bd38a"} Jul 12 18:25:32.434032 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.433999 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" event={"ID":"e0391cb3-5bc7-4fb5-9d22-f831b4125304","Type":"ContainerStarted","Data":"f2b3275981aa8d923084d3c290b5ce9c21dd0ecb9b5d1813a5b41f3575d674fa"} Jul 12 18:25:32.434158 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.434093 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" event={"ID":"e0391cb3-5bc7-4fb5-9d22-f831b4125304","Type":"ContainerStarted","Data":"52bf0e69d360a21b7b1c07e38bfa7365d10bdc6479b308ac16572cd76198d6a3"} Jul 12 18:25:32.434158 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.434143 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" event={"ID":"e0391cb3-5bc7-4fb5-9d22-f831b4125304","Type":"ContainerStarted","Data":"d9800d2bf2e2787ccbd4ba0580efc03703e0f94ca642f0f8145f0fa5824e944b"} Jul 12 18:25:32.434266 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.434189 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" event={"ID":"e0391cb3-5bc7-4fb5-9d22-f831b4125304","Type":"ContainerStarted","Data":"85c2d7205f9b4b9e8e44290ea007c53b93af6b674cf8583ba24939c3294f1df7"} Jul 12 18:25:32.435518 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.435491 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5dzg6" event={"ID":"64c451f1-c731-40c5-a837-b515e65009e0","Type":"ContainerStarted","Data":"a5f667b0dfa4115983251e0693a54d4852d0e3dcd096556a763acead559e03c3"} Jul 12 18:25:32.436983 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.436958 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" event={"ID":"a62e223b-ddd1-4dd2-92e6-c87f9b606ece","Type":"ContainerStarted","Data":"1d358c4d36e2d07bd2e1840b2b0b12caca694e9fed4acc6a5065cfe08526f8d2"} Jul 12 18:25:32.438301 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.438277 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cwkh5" event={"ID":"5a8d8139-c043-4f8c-b49e-76a54608b5af","Type":"ContainerStarted","Data":"e4965a48d4d3d234063aafad8f300440c4c99b7a2069a2339c0d786fdd872b99"} Jul 12 18:25:32.439825 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.439802 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x7b84" event={"ID":"fa50005b-ad03-474a-a648-f9fde6a932a6","Type":"ContainerStarted","Data":"c0c7bd6f3b33622f1e920b971d098d1fb7510d12dfd6064c79c7dfa116739427"} Jul 12 18:25:32.441374 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.441352 2585 generic.go:356] "Generic (PLEG): container finished" podID="dd93dddb-eaaa-470d-84a4-17c4f61493e8" containerID="1355a60f31d6b2b8c77e37ab9934781e335f9dc09d0764bb623dd704c90682e3" exitCode=0 Jul 12 18:25:32.441474 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.441431 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llfzt" event={"ID":"dd93dddb-eaaa-470d-84a4-17c4f61493e8","Type":"ContainerDied","Data":"1355a60f31d6b2b8c77e37ab9934781e335f9dc09d0764bb623dd704c90682e3"} Jul 12 18:25:32.443043 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.443016 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" event={"ID":"ecd4d59c-8720-4481-ad1b-f4943dff29a4","Type":"ContainerStarted","Data":"a4c1adc31a59f40bcdcb67958cd3298dcde48b7d72051370e0f2731eec59e8aa"} Jul 12 18:25:32.457920 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.457875 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9ph4f" podStartSLOduration=2.114339789 podStartE2EDuration="22.457859067s" totalImagesPullingTime="20.343519278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:32.457700896 +0000 UTC m=+23.657167925" watchObservedRunningTime="2026-07-12 18:25:32.457859067 +0000 UTC m=+23.657326097" Jul 12 18:25:32.458315 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.458289 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-170.ec2.internal" podStartSLOduration=21.458281502 podStartE2EDuration="21.458281502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:14.397055752 +0000 UTC m=+5.596522790" watchObservedRunningTime="2026-07-12 18:25:32.458281502 +0000 UTC m=+23.657748534" Jul 12 18:25:32.479102 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.479058 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5s6z5" podStartSLOduration=3.373453791 podStartE2EDuration="22.479046699s" totalImagesPullingTime="19.105592908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:32.478390323 +0000 UTC m=+23.677857352" watchObservedRunningTime="2026-07-12 18:25:32.479046699 +0000 UTC m=+23.678513729" Jul 12 18:25:32.563527 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.563473 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cwkh5" podStartSLOduration=6.512571636 podStartE2EDuration="22.563455281s" totalImagesPullingTime="16.050883645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:32.531674009 +0000 UTC m=+23.731141039" watchObservedRunningTime="2026-07-12 18:25:32.563455281 +0000 UTC m=+23.762922311" Jul 12 18:25:32.563738 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.563612 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-x7b84" podStartSLOduration=3.463031764 podStartE2EDuration="22.563606476s" totalImagesPullingTime="19.100574712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:32.562891928 +0000 UTC m=+23.762358958" watchObservedRunningTime="2026-07-12 18:25:32.563606476 +0000 UTC m=+23.763073505" Jul 12 18:25:32.593309 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:32.593187 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5dzg6" podStartSLOduration=7.716804172 podStartE2EDuration="22.593163593s" totalImagesPullingTime="14.876359421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:32.593058897 +0000 UTC m=+23.792525929" watchObservedRunningTime="2026-07-12 18:25:32.593163593 +0000 UTC m=+23.792630624" Jul 12 18:25:33.298713 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:33.298678 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:33.298861 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:33.298686 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:33.298861 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:33.298686 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:33.298861 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:33.298815 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:33.299039 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:33.298889 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:33.299039 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:33.298991 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:33.408326 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:33.408293 2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 12 18:25:33.448804 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:33.448769 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" event={"ID":"e0391cb3-5bc7-4fb5-9d22-f831b4125304","Type":"ContainerStarted","Data":"9c416a9552ae96c76ee177f6b2f60af54ca4ac737ea0102bf27ebe8adf800f90"} Jul 12 18:25:33.449236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:33.448854 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" event={"ID":"e0391cb3-5bc7-4fb5-9d22-f831b4125304","Type":"ContainerStarted","Data":"bb910633664df97182cfcd33526b2f9f32f3cde16069f2b86604bb11356c703f"} Jul 12 18:25:33.450091 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:33.450066 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bx895" event={"ID":"aae3b7fb-6dd9-4a5f-9ada-88a9b9e3663e","Type":"ContainerStarted","Data":"23b465e0e498d8c9412e69de614e21e7bab43ac076e2052e3d503c4f1980d2b8"} Jul 12 18:25:33.451862 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:33.451836 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" event={"ID":"a62e223b-ddd1-4dd2-92e6-c87f9b606ece","Type":"ContainerStarted","Data":"7b9f9a10dd2c742d7209e56a2dadb4b4c8d5c39ac201bdd4fe18a9bf83d2aa10"} Jul 12 18:25:34.316719 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:34.316597 2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T18:25:33.408318303Z","UUID":"c981bef1-c415-4e19-84ac-8872cf10c997","Handler":null,"Name":"","Endpoint":""} Jul 12 18:25:34.318513 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:34.318485 2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 12 18:25:34.318648 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:34.318532 2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 12 18:25:35.299440 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:35.298998 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:35.299440 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:35.299045 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:35.299440 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:35.298998 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:35.299440 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:35.299139 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:35.299440 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:35.299242 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:35.299440 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:35.299311 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:35.460889 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:35.460843 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" event={"ID":"e0391cb3-5bc7-4fb5-9d22-f831b4125304","Type":"ContainerStarted","Data":"628436facfeddaa2de7c18851d8f867de37932a3250f7ded160a54f33a9e9525"} Jul 12 18:25:35.463327 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:35.463292 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" event={"ID":"a62e223b-ddd1-4dd2-92e6-c87f9b606ece","Type":"ContainerStarted","Data":"4433ac60e45c5230807271edb7483295c0936f25f74592744fa607b3fbce011c"} Jul 12 18:25:35.481780 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:35.481714 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s47kq" podStartSLOduration=3.577399262 podStartE2EDuration="25.481690616s" totalImagesPullingTime="21.904291354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:35.481510858 +0000 UTC m=+26.680977888" watchObservedRunningTime="2026-07-12 18:25:35.481690616 +0000 UTC m=+26.681157647" Jul 12 18:25:35.482699 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:35.482652 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bx895" podStartSLOduration=6.374357476 podStartE2EDuration="25.482639408s" totalImagesPullingTime="19.108281932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:33.464390259 +0000 UTC m=+24.663857290" watchObservedRunningTime="2026-07-12 18:25:35.482639408 +0000 UTC m=+26.682106488" Jul 12 18:25:36.507461 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:36.507415 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cwkh5" Jul 12 18:25:36.508258 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:36.508232 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cwkh5" Jul 12 18:25:37.298825 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:37.298577 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:37.299018 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:37.298577 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:37.299018 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:37.298953 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:37.299018 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:37.298577 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:37.299164 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:37.299033 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:37.299164 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:37.299114 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:37.467749 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:37.467713 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cwkh5" Jul 12 18:25:37.468255 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:37.468237 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cwkh5" Jul 12 18:25:38.472091 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:38.471874 2585 generic.go:356] "Generic (PLEG): container finished" podID="dd93dddb-eaaa-470d-84a4-17c4f61493e8" containerID="4e7a183af4892afe0b8916e9744fd61e03d4c77a0cbbd997b8d9b30b0ec30237" exitCode=0 Jul 12 18:25:38.472956 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:38.471939 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llfzt" event={"ID":"dd93dddb-eaaa-470d-84a4-17c4f61493e8","Type":"ContainerDied","Data":"4e7a183af4892afe0b8916e9744fd61e03d4c77a0cbbd997b8d9b30b0ec30237"} Jul 12 18:25:38.477809 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:38.477776 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" event={"ID":"e0391cb3-5bc7-4fb5-9d22-f831b4125304","Type":"ContainerStarted","Data":"dfdeadb077217d86c8bbb9aa39dc633c12888d9e244547f9cb45eb756e7e8d26"} Jul 12 18:25:38.517028 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:38.516972 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" podStartSLOduration=9.344383356 podStartE2EDuration="28.516955634s" totalImagesPullingTime="19.172572278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:38.515639809 +0000 UTC m=+29.715106838" watchObservedRunningTime="2026-07-12 18:25:38.516955634 +0000 UTC m=+29.716422660" Jul 12 18:25:39.299922 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:39.299880 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:39.300236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:39.300010 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:39.300236 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:39.300026 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:39.300236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:39.300082 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:39.300236 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:39.300160 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:39.300236 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:39.300225 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:39.481265 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:39.481234 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:39.481689 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:39.481315 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:39.481689 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:39.481362 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:39.497151 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:39.497127 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:39.497302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:39.497241 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:25:40.030370 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:40.030334 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j5bpc"] Jul 12 18:25:40.030521 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:40.030448 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:40.030557 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:40.030535 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:40.033510 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:40.033457 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8q6bw"] Jul 12 18:25:40.033668 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:40.033594 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:40.033734 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:40.033701 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:40.034168 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:40.034148 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rgkpj"] Jul 12 18:25:40.034255 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:40.034229 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:40.034329 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:40.034312 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:40.484617 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:40.484532 2585 generic.go:356] "Generic (PLEG): container finished" podID="dd93dddb-eaaa-470d-84a4-17c4f61493e8" containerID="272d9e16e2e375ac672cf9c15d1f1eab7c972e80b61cb61e4931315209b55b2d" exitCode=0 Jul 12 18:25:40.485032 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:40.484621 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llfzt" event={"ID":"dd93dddb-eaaa-470d-84a4-17c4f61493e8","Type":"ContainerDied","Data":"272d9e16e2e375ac672cf9c15d1f1eab7c972e80b61cb61e4931315209b55b2d"} Jul 12 18:25:41.298376 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:41.298343 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:41.298565 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:41.298468 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:42.298392 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:42.298355 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:42.298803 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:42.298355 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:42.298803 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:42.298480 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:42.298803 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:42.298531 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:42.491824 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:42.491793 2585 generic.go:356] "Generic (PLEG): container finished" podID="dd93dddb-eaaa-470d-84a4-17c4f61493e8" containerID="4626f0b93923dc671c65ab8f8138f24f9dc4849e2190776c9601c5bdfdf9d6d5" exitCode=0 Jul 12 18:25:42.491824 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:42.491831 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llfzt" event={"ID":"dd93dddb-eaaa-470d-84a4-17c4f61493e8","Type":"ContainerDied","Data":"4626f0b93923dc671c65ab8f8138f24f9dc4849e2190776c9601c5bdfdf9d6d5"} Jul 12 18:25:42.990410 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:42.990371 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs\") pod \"network-metrics-daemon-rgkpj\" (UID: \"87644bbc-d561-4ca5-8b2b-38bf70f01efb\") " pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:42.990573 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:42.990534 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:25:42.990638 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:42.990599 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs podName:87644bbc-d561-4ca5-8b2b-38bf70f01efb nodeName:}" failed. No retries permitted until 2026-07-12 18:26:14.990581083 +0000 UTC m=+66.190048103 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs") pod "network-metrics-daemon-rgkpj" (UID: "87644bbc-d561-4ca5-8b2b-38bf70f01efb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:25:43.192058 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:43.191948 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz8h8\" (UniqueName: \"kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8\") pod \"network-check-target-8q6bw\" (UID: \"7abc23a1-1bde-4046-a60d-cf3e93e64f63\") " pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:43.192058 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:43.192004 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:43.192292 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:43.192112 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 18:25:43.192292 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:43.192131 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 18:25:43.192292 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:43.192159 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 18:25:43.192292 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:43.192166 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret podName:6ad0a79f-69ee-49e8-90eb-c27482619b69 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:15.192151631 +0000 UTC m=+66.391618640 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret") pod "global-pull-secret-syncer-j5bpc" (UID: "6ad0a79f-69ee-49e8-90eb-c27482619b69") : object "kube-system"/"original-pull-secret" not registered Jul 12 18:25:43.192292 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:43.192172 2585 projected.go:196] Error preparing data for projected volume kube-api-access-wz8h8 for pod openshift-network-diagnostics/network-check-target-8q6bw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:25:43.192292 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:43.192222 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8 podName:7abc23a1-1bde-4046-a60d-cf3e93e64f63 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:15.192206591 +0000 UTC m=+66.391673606 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wz8h8" (UniqueName: "kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8") pod "network-check-target-8q6bw" (UID: "7abc23a1-1bde-4046-a60d-cf3e93e64f63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:25:43.298577 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:43.298537 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:43.299053 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:43.298672 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgkpj" podUID="87644bbc-d561-4ca5-8b2b-38bf70f01efb" Jul 12 18:25:44.299083 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.299045 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:44.299557 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.299091 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:44.299557 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:44.299178 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j5bpc" podUID="6ad0a79f-69ee-49e8-90eb-c27482619b69" Jul 12 18:25:44.299557 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:44.299300 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8q6bw" podUID="7abc23a1-1bde-4046-a60d-cf3e93e64f63" Jul 12 18:25:44.534832 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.534806 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-170.ec2.internal" event="NodeReady" Jul 12 18:25:44.535006 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.534946 2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 12 18:25:44.569273 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.569189 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67bd449f9f-lf78m"] Jul 12 18:25:44.572830 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.572810 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.575134 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.575111 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 12 18:25:44.575357 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.575336 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 12 18:25:44.575450 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.575345 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-twcn4\"" Jul 12 18:25:44.575535 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.575512 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 12 18:25:44.580757 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.580734 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 12 18:25:44.585205 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.585179 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lphwv"] Jul 12 18:25:44.589610 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.589285 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67bd449f9f-lf78m"] Jul 12 18:25:44.589610 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.589397 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:44.590199 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.590177 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-l98hb"] Jul 12 18:25:44.591722 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.591700 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 12 18:25:44.591843 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.591784 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 12 18:25:44.591908 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.591788 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7m82z\"" Jul 12 18:25:44.594101 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.594080 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l98hb" Jul 12 18:25:44.596327 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.596304 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 12 18:25:44.596453 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.596426 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-m2nlf\"" Jul 12 18:25:44.596524 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.596460 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 12 18:25:44.596524 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.596516 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 12 18:25:44.600758 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.600737 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lphwv"] Jul 12 18:25:44.603880 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.603856 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l98hb"] Jul 12 18:25:44.702441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.702395 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-bound-sa-token\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.702594 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.702458 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cfdb85db-878a-43b4-af66-ebe34aea186a-installation-pull-secrets\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.702594 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.702485 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9aef0f72-416e-4c37-98b8-a68e30a65ce6-cert\") pod \"ingress-canary-l98hb\" (UID: \"9aef0f72-416e-4c37-98b8-a68e30a65ce6\") " pod="openshift-ingress-canary/ingress-canary-l98hb" Jul 12 18:25:44.702594 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.702518 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb62dd7e-0d0f-4128-87dd-3158aa45a396-config-volume\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:44.702594 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.702533 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cfdb85db-878a-43b4-af66-ebe34aea186a-trusted-ca\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.702594 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.702551 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrfxd\" (UniqueName: \"kubernetes.io/projected/9aef0f72-416e-4c37-98b8-a68e30a65ce6-kube-api-access-vrfxd\") pod \"ingress-canary-l98hb\" (UID: \"9aef0f72-416e-4c37-98b8-a68e30a65ce6\") " pod="openshift-ingress-canary/ingress-canary-l98hb" Jul 12 18:25:44.702594 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.702582 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:44.702594 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.702596 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb62dd7e-0d0f-4128-87dd-3158aa45a396-tmp-dir\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:44.702817 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.702611 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cfdb85db-878a-43b4-af66-ebe34aea186a-image-registry-private-configuration\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.702817 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.702635 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cfdb85db-878a-43b4-af66-ebe34aea186a-ca-trust-extracted\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.702817 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.702660 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwnfl\" (UniqueName: \"kubernetes.io/projected/cb62dd7e-0d0f-4128-87dd-3158aa45a396-kube-api-access-rwnfl\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:44.702817 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.702678 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.702817 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.702694 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-certificates\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.702817 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.702738 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdkrb\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-kube-api-access-bdkrb\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.803797 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.803764 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwnfl\" (UniqueName: \"kubernetes.io/projected/cb62dd7e-0d0f-4128-87dd-3158aa45a396-kube-api-access-rwnfl\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:44.803797 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.803799 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.804061 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.803826 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-certificates\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.804061 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:44.803960 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 18:25:44.804061 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:44.803983 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bd449f9f-lf78m: secret "image-registry-tls" not found Jul 12 18:25:44.804061 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:44.804050 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls podName:cfdb85db-878a-43b4-af66-ebe34aea186a nodeName:}" failed. No retries permitted until 2026-07-12 18:25:45.304029532 +0000 UTC m=+36.503496561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls") pod "image-registry-67bd449f9f-lf78m" (UID: "cfdb85db-878a-43b4-af66-ebe34aea186a") : secret "image-registry-tls" not found Jul 12 18:25:44.804248 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.803962 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdkrb\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-kube-api-access-bdkrb\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.804518 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.804484 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-bound-sa-token\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.805026 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.804586 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cfdb85db-878a-43b4-af66-ebe34aea186a-installation-pull-secrets\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.805026 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.804792 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9aef0f72-416e-4c37-98b8-a68e30a65ce6-cert\") pod \"ingress-canary-l98hb\" (UID: \"9aef0f72-416e-4c37-98b8-a68e30a65ce6\") " pod="openshift-ingress-canary/ingress-canary-l98hb" Jul 12 18:25:44.805206 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.804867 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb62dd7e-0d0f-4128-87dd-3158aa45a396-config-volume\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:44.805206 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.805116 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cfdb85db-878a-43b4-af66-ebe34aea186a-trusted-ca\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.805206 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.805152 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrfxd\" (UniqueName: \"kubernetes.io/projected/9aef0f72-416e-4c37-98b8-a68e30a65ce6-kube-api-access-vrfxd\") pod \"ingress-canary-l98hb\" (UID: \"9aef0f72-416e-4c37-98b8-a68e30a65ce6\") " pod="openshift-ingress-canary/ingress-canary-l98hb" Jul 12 18:25:44.805347 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.805210 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:44.805347 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.805237 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb62dd7e-0d0f-4128-87dd-3158aa45a396-tmp-dir\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:44.805347 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.805279 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-certificates\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.805477 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.805401 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cfdb85db-878a-43b4-af66-ebe34aea186a-image-registry-private-configuration\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.805477 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.805440 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cfdb85db-878a-43b4-af66-ebe34aea186a-ca-trust-extracted\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.806276 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.805816 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cfdb85db-878a-43b4-af66-ebe34aea186a-ca-trust-extracted\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.806276 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.805827 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb62dd7e-0d0f-4128-87dd-3158aa45a396-config-volume\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:44.806276 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.806092 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb62dd7e-0d0f-4128-87dd-3158aa45a396-tmp-dir\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:44.806276 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:44.806252 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 18:25:44.806521 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:44.806309 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls podName:cb62dd7e-0d0f-4128-87dd-3158aa45a396 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:45.306287216 +0000 UTC m=+36.505754239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls") pod "dns-default-lphwv" (UID: "cb62dd7e-0d0f-4128-87dd-3158aa45a396") : secret "dns-default-metrics-tls" not found Jul 12 18:25:44.809291 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.808623 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cfdb85db-878a-43b4-af66-ebe34aea186a-trusted-ca\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.810428 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.810401 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cfdb85db-878a-43b4-af66-ebe34aea186a-installation-pull-secrets\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.811098 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.811073 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9aef0f72-416e-4c37-98b8-a68e30a65ce6-cert\") pod \"ingress-canary-l98hb\" (UID: \"9aef0f72-416e-4c37-98b8-a68e30a65ce6\") " pod="openshift-ingress-canary/ingress-canary-l98hb" Jul 12 18:25:44.811392 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.811370 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cfdb85db-878a-43b4-af66-ebe34aea186a-image-registry-private-configuration\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.813025 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.813006 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwnfl\" (UniqueName: \"kubernetes.io/projected/cb62dd7e-0d0f-4128-87dd-3158aa45a396-kube-api-access-rwnfl\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:44.815651 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.815586 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-bound-sa-token\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.815804 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.815783 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdkrb\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-kube-api-access-bdkrb\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:44.816910 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.816889 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrfxd\" (UniqueName: \"kubernetes.io/projected/9aef0f72-416e-4c37-98b8-a68e30a65ce6-kube-api-access-vrfxd\") pod \"ingress-canary-l98hb\" (UID: \"9aef0f72-416e-4c37-98b8-a68e30a65ce6\") " pod="openshift-ingress-canary/ingress-canary-l98hb" Jul 12 18:25:44.908738 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:44.908655 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l98hb" Jul 12 18:25:45.062706 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:45.062674 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l98hb"] Jul 12 18:25:45.067110 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:45.067076 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aef0f72_416e_4c37_98b8_a68e30a65ce6.slice/crio-91228b2dc0d86c37689038ea0bfac3c85324d65c7a43a643cbf1d59f86ef57dd WatchSource:0}: Error finding container 91228b2dc0d86c37689038ea0bfac3c85324d65c7a43a643cbf1d59f86ef57dd: Status 404 returned error can't find the container with id 91228b2dc0d86c37689038ea0bfac3c85324d65c7a43a643cbf1d59f86ef57dd Jul 12 18:25:45.299230 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:45.299194 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:25:45.301589 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:45.301566 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 18:25:45.301721 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:45.301598 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gx6mz\"" Jul 12 18:25:45.310133 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:45.310106 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:45.310247 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:45.310146 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:45.310286 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:45.310256 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 18:25:45.310286 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:45.310276 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 18:25:45.310347 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:45.310287 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bd449f9f-lf78m: secret "image-registry-tls" not found Jul 12 18:25:45.310347 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:45.310319 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls podName:cb62dd7e-0d0f-4128-87dd-3158aa45a396 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:46.310300985 +0000 UTC m=+37.509768000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls") pod "dns-default-lphwv" (UID: "cb62dd7e-0d0f-4128-87dd-3158aa45a396") : secret "dns-default-metrics-tls" not found Jul 12 18:25:45.310412 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:45.310366 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls podName:cfdb85db-878a-43b4-af66-ebe34aea186a nodeName:}" failed. No retries permitted until 2026-07-12 18:25:46.310356169 +0000 UTC m=+37.509823192 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls") pod "image-registry-67bd449f9f-lf78m" (UID: "cfdb85db-878a-43b4-af66-ebe34aea186a") : secret "image-registry-tls" not found Jul 12 18:25:45.502656 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:45.502617 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l98hb" event={"ID":"9aef0f72-416e-4c37-98b8-a68e30a65ce6","Type":"ContainerStarted","Data":"91228b2dc0d86c37689038ea0bfac3c85324d65c7a43a643cbf1d59f86ef57dd"} Jul 12 18:25:46.298399 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:46.298127 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:25:46.298950 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:46.298183 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:25:46.301278 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:46.301253 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 18:25:46.301656 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:46.301255 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 18:25:46.302108 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:46.302053 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 18:25:46.302108 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:46.302054 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p8prd\"" Jul 12 18:25:46.319065 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:46.318956 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:46.319065 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:46.319012 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:46.319065 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:46.319065 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 18:25:46.319328 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:46.319124 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls podName:cb62dd7e-0d0f-4128-87dd-3158aa45a396 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:48.319106191 +0000 UTC m=+39.518573214 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls") pod "dns-default-lphwv" (UID: "cb62dd7e-0d0f-4128-87dd-3158aa45a396") : secret "dns-default-metrics-tls" not found Jul 12 18:25:46.319328 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:46.319155 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 18:25:46.319328 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:46.319166 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bd449f9f-lf78m: secret "image-registry-tls" not found Jul 12 18:25:46.319328 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:46.319203 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls podName:cfdb85db-878a-43b4-af66-ebe34aea186a nodeName:}" failed. No retries permitted until 2026-07-12 18:25:48.319191013 +0000 UTC m=+39.518658022 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls") pod "image-registry-67bd449f9f-lf78m" (UID: "cfdb85db-878a-43b4-af66-ebe34aea186a") : secret "image-registry-tls" not found Jul 12 18:25:48.335946 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.335901 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:48.336419 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.335959 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:48.336419 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:48.336066 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 18:25:48.336419 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:48.336077 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bd449f9f-lf78m: secret "image-registry-tls" not found Jul 12 18:25:48.336419 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:48.336073 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 18:25:48.336419 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:48.336128 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls podName:cfdb85db-878a-43b4-af66-ebe34aea186a nodeName:}" failed. No retries permitted until 2026-07-12 18:25:52.336113291 +0000 UTC m=+43.535580318 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls") pod "image-registry-67bd449f9f-lf78m" (UID: "cfdb85db-878a-43b4-af66-ebe34aea186a") : secret "image-registry-tls" not found Jul 12 18:25:48.336419 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:48.336141 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls podName:cb62dd7e-0d0f-4128-87dd-3158aa45a396 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:52.33613588 +0000 UTC m=+43.535602888 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls") pod "dns-default-lphwv" (UID: "cb62dd7e-0d0f-4128-87dd-3158aa45a396") : secret "dns-default-metrics-tls" not found Jul 12 18:25:48.360634 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.360594 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9cft2"] Jul 12 18:25:48.366021 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.365994 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9cft2" Jul 12 18:25:48.368416 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.368354 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 12 18:25:48.369146 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.369128 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 12 18:25:48.369282 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.369210 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-5pl56\"" Jul 12 18:25:48.370687 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.370653 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9cft2"] Jul 12 18:25:48.373849 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.373630 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-kzl7d"] Jul 12 18:25:48.377071 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.377052 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" Jul 12 18:25:48.379422 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.379403 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 12 18:25:48.379422 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.379415 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 12 18:25:48.379584 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.379408 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-k79gm\"" Jul 12 18:25:48.387716 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.387692 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-kzl7d"] Jul 12 18:25:48.436504 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.436465 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9z74\" (UniqueName: \"kubernetes.io/projected/ca704d74-c4dc-4547-b226-2f910224db6c-kube-api-access-c9z74\") pod \"volume-data-source-validator-78495597f6-9cft2\" (UID: \"ca704d74-c4dc-4547-b226-2f910224db6c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9cft2" Jul 12 18:25:48.464047 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.464014 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf"] Jul 12 18:25:48.466977 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.466952 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" Jul 12 18:25:48.467413 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.467391 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k"] Jul 12 18:25:48.469299 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.469275 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 12 18:25:48.469389 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.469280 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qvvn7\"" Jul 12 18:25:48.469389 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.469377 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 12 18:25:48.469496 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.469415 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 12 18:25:48.470611 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.470595 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-96sxt"] Jul 12 18:25:48.470728 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.470713 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" Jul 12 18:25:48.473123 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.473103 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 12 18:25:48.473322 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.473300 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 12 18:25:48.473322 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.473318 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 12 18:25:48.473440 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.473421 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-gqg5q\"" Jul 12 18:25:48.473567 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.473547 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 12 18:25:48.474385 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.474290 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.476648 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.476628 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 12 18:25:48.476765 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.476629 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 12 18:25:48.476895 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.476878 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 12 18:25:48.476991 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.476904 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 12 18:25:48.476991 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.476973 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf"] Jul 12 18:25:48.477075 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.477006 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-nfnv6\"" Jul 12 18:25:48.482589 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.482566 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 12 18:25:48.485515 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.485380 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-96sxt"] Jul 12 18:25:48.486473 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.486447 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-549c4cff84-xqpgj"] Jul 12 18:25:48.490707 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.490686 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:48.492694 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.492671 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k"] Jul 12 18:25:48.493159 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.493139 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 12 18:25:48.493353 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.493337 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 12 18:25:48.493437 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.493423 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 12 18:25:48.493721 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.493703 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 12 18:25:48.493869 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.493764 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-z26dw\"" Jul 12 18:25:48.493869 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.493812 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 12 18:25:48.493869 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.493830 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 12 18:25:48.501280 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.501260 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-549c4cff84-xqpgj"] Jul 12 18:25:48.513738 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.513705 2585 generic.go:356] "Generic (PLEG): container finished" podID="dd93dddb-eaaa-470d-84a4-17c4f61493e8" containerID="dc414c8174ef3dcc037689b5b4b0ed3f2b9e770d383dad5f5c24e642a868f0f1" exitCode=0 Jul 12 18:25:48.513882 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.513787 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llfzt" event={"ID":"dd93dddb-eaaa-470d-84a4-17c4f61493e8","Type":"ContainerDied","Data":"dc414c8174ef3dcc037689b5b4b0ed3f2b9e770d383dad5f5c24e642a868f0f1"} Jul 12 18:25:48.515325 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.515299 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l98hb" event={"ID":"9aef0f72-416e-4c37-98b8-a68e30a65ce6","Type":"ContainerStarted","Data":"2180c620f155bd088c0e975f7f4fa8efe25a09359afd0361928fe8f1e7e27be7"} Jul 12 18:25:48.537749 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.537718 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8m5c\" (UniqueName: \"kubernetes.io/projected/621b603b-b0fd-4925-bbc6-015e3fd4d097-kube-api-access-l8m5c\") pod \"service-ca-operator-7695fc6c8c-mqv6k\" (UID: \"621b603b-b0fd-4925-bbc6-015e3fd4d097\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" Jul 12 18:25:48.537872 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.537837 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9z74\" (UniqueName: \"kubernetes.io/projected/ca704d74-c4dc-4547-b226-2f910224db6c-kube-api-access-c9z74\") pod \"volume-data-source-validator-78495597f6-9cft2\" (UID: \"ca704d74-c4dc-4547-b226-2f910224db6c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9cft2" Jul 12 18:25:48.537972 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.537876 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2875\" (UniqueName: \"kubernetes.io/projected/b5d716f4-9338-4351-a5fc-46b1513626d0-kube-api-access-m2875\") pod \"cluster-samples-operator-65db858894-qrltf\" (UID: \"b5d716f4-9338-4351-a5fc-46b1513626d0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" Jul 12 18:25:48.537972 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.537905 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/621b603b-b0fd-4925-bbc6-015e3fd4d097-serving-cert\") pod \"service-ca-operator-7695fc6c8c-mqv6k\" (UID: \"621b603b-b0fd-4925-bbc6-015e3fd4d097\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" Jul 12 18:25:48.537972 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.537960 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-kzl7d\" (UID: \"d1e3d19c-3412-49a9-a51f-4658c02209bd\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" Jul 12 18:25:48.538128 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.538024 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d1e3d19c-3412-49a9-a51f-4658c02209bd-nginx-conf\") pod \"networking-console-plugin-6cf856758c-kzl7d\" (UID: \"d1e3d19c-3412-49a9-a51f-4658c02209bd\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" Jul 12 18:25:48.538128 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.538067 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-qrltf\" (UID: \"b5d716f4-9338-4351-a5fc-46b1513626d0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" Jul 12 18:25:48.538128 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.538103 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/621b603b-b0fd-4925-bbc6-015e3fd4d097-config\") pod \"service-ca-operator-7695fc6c8c-mqv6k\" (UID: \"621b603b-b0fd-4925-bbc6-015e3fd4d097\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" Jul 12 18:25:48.553598 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.553565 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9z74\" (UniqueName: \"kubernetes.io/projected/ca704d74-c4dc-4547-b226-2f910224db6c-kube-api-access-c9z74\") pod \"volume-data-source-validator-78495597f6-9cft2\" (UID: \"ca704d74-c4dc-4547-b226-2f910224db6c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9cft2" Jul 12 18:25:48.564563 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.564521 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-8sjbq"] Jul 12 18:25:48.566541 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.566485 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-l98hb" podStartSLOduration=1.5166157729999998 podStartE2EDuration="4.566441127s" totalImagesPullingTime="3.049825354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:48.563884869 +0000 UTC m=+39.763351901" watchObservedRunningTime="2026-07-12 18:25:48.566441127 +0000 UTC m=+39.765908157" Jul 12 18:25:48.583860 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.583829 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2"] Jul 12 18:25:48.584024 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.583972 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8sjbq" Jul 12 18:25:48.586709 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.586688 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-6nj2v\"" Jul 12 18:25:48.610689 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.610662 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-9nvfk"] Jul 12 18:25:48.610833 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.610798 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:25:48.615139 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.615112 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lz5wn\"" Jul 12 18:25:48.615299 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.615119 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 12 18:25:48.615370 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.615338 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 12 18:25:48.616745 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.616726 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 12 18:25:48.619006 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.618959 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 12 18:25:48.635016 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.634984 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6"] Jul 12 18:25:48.635167 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.635104 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:48.637721 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.637701 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 12 18:25:48.637844 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.637786 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 12 18:25:48.637844 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.637809 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-mvplg\"" Jul 12 18:25:48.637986 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.637968 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 12 18:25:48.638059 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.638016 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 12 18:25:48.638713 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.638669 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrn7d\" (UniqueName: \"kubernetes.io/projected/80edfd93-3c7a-4356-9c21-6b1376df14fb-kube-api-access-nrn7d\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:48.638853 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.638837 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:48.638976 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.638893 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-tmp\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.638976 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.638965 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2875\" (UniqueName: \"kubernetes.io/projected/b5d716f4-9338-4351-a5fc-46b1513626d0-kube-api-access-m2875\") pod \"cluster-samples-operator-65db858894-qrltf\" (UID: \"b5d716f4-9338-4351-a5fc-46b1513626d0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" Jul 12 18:25:48.639118 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.638990 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.639118 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.639017 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-kzl7d\" (UID: \"d1e3d19c-3412-49a9-a51f-4658c02209bd\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" Jul 12 18:25:48.639118 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.639054 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-qrltf\" (UID: \"b5d716f4-9338-4351-a5fc-46b1513626d0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" Jul 12 18:25:48.639118 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.639081 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8m5c\" (UniqueName: \"kubernetes.io/projected/621b603b-b0fd-4925-bbc6-015e3fd4d097-kube-api-access-l8m5c\") pod \"service-ca-operator-7695fc6c8c-mqv6k\" (UID: \"621b603b-b0fd-4925-bbc6-015e3fd4d097\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" Jul 12 18:25:48.639118 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.639110 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/621b603b-b0fd-4925-bbc6-015e3fd4d097-config\") pod \"service-ca-operator-7695fc6c8c-mqv6k\" (UID: \"621b603b-b0fd-4925-bbc6-015e3fd4d097\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" Jul 12 18:25:48.639341 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.639200 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.639341 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.639233 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-stats-auth\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:48.639341 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.639259 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:48.639341 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.639290 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/621b603b-b0fd-4925-bbc6-015e3fd4d097-serving-cert\") pod \"service-ca-operator-7695fc6c8c-mqv6k\" (UID: \"621b603b-b0fd-4925-bbc6-015e3fd4d097\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" Jul 12 18:25:48.639586 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.639381 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-snapshots\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.639672 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:48.639651 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 18:25:48.639726 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:48.639712 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls podName:b5d716f4-9338-4351-a5fc-46b1513626d0 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:49.139692412 +0000 UTC m=+40.339159438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls") pod "cluster-samples-operator-65db858894-qrltf" (UID: "b5d716f4-9338-4351-a5fc-46b1513626d0") : secret "samples-operator-tls" not found Jul 12 18:25:48.639726 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:48.639717 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 18:25:48.639848 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.639787 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-default-certificate\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:48.639949 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.639910 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkqhv\" (UniqueName: \"kubernetes.io/projected/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-kube-api-access-pkqhv\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.640018 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:48.639974 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert podName:d1e3d19c-3412-49a9-a51f-4658c02209bd nodeName:}" failed. No retries permitted until 2026-07-12 18:25:49.139955532 +0000 UTC m=+40.339422541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-kzl7d" (UID: "d1e3d19c-3412-49a9-a51f-4658c02209bd") : secret "networking-console-plugin-cert" not found Jul 12 18:25:48.640118 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.640103 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-serving-cert\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.640177 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.640161 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/621b603b-b0fd-4925-bbc6-015e3fd4d097-config\") pod \"service-ca-operator-7695fc6c8c-mqv6k\" (UID: \"621b603b-b0fd-4925-bbc6-015e3fd4d097\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" Jul 12 18:25:48.640177 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.640167 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d1e3d19c-3412-49a9-a51f-4658c02209bd-nginx-conf\") pod \"networking-console-plugin-6cf856758c-kzl7d\" (UID: \"d1e3d19c-3412-49a9-a51f-4658c02209bd\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" Jul 12 18:25:48.640842 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.640818 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d1e3d19c-3412-49a9-a51f-4658c02209bd-nginx-conf\") pod \"networking-console-plugin-6cf856758c-kzl7d\" (UID: \"d1e3d19c-3412-49a9-a51f-4658c02209bd\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" Jul 12 18:25:48.642558 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.642536 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/621b603b-b0fd-4925-bbc6-015e3fd4d097-serving-cert\") pod \"service-ca-operator-7695fc6c8c-mqv6k\" (UID: \"621b603b-b0fd-4925-bbc6-015e3fd4d097\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" Jul 12 18:25:48.643250 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.643231 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 12 18:25:48.655228 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.655204 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2"] Jul 12 18:25:48.655349 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.655244 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-8sjbq"] Jul 12 18:25:48.655349 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.655271 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-9nvfk"] Jul 12 18:25:48.655349 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.655306 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6"] Jul 12 18:25:48.655514 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.655354 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" Jul 12 18:25:48.658823 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.658682 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 12 18:25:48.658823 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.658712 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 12 18:25:48.658823 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.658715 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 12 18:25:48.658823 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.658803 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-p7pht\"" Jul 12 18:25:48.659419 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.659048 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 12 18:25:48.659507 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.659489 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2875\" (UniqueName: \"kubernetes.io/projected/b5d716f4-9338-4351-a5fc-46b1513626d0-kube-api-access-m2875\") pod \"cluster-samples-operator-65db858894-qrltf\" (UID: \"b5d716f4-9338-4351-a5fc-46b1513626d0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" Jul 12 18:25:48.659609 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.659588 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8m5c\" (UniqueName: \"kubernetes.io/projected/621b603b-b0fd-4925-bbc6-015e3fd4d097-kube-api-access-l8m5c\") pod \"service-ca-operator-7695fc6c8c-mqv6k\" (UID: \"621b603b-b0fd-4925-bbc6-015e3fd4d097\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" Jul 12 18:25:48.676245 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.676064 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9cft2" Jul 12 18:25:48.741031 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741001 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrn7d\" (UniqueName: \"kubernetes.io/projected/80edfd93-3c7a-4356-9c21-6b1376df14fb-kube-api-access-nrn7d\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:48.741217 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741037 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:48.741217 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741059 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-tmp\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.741338 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:48.741220 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle podName:80edfd93-3c7a-4356-9c21-6b1376df14fb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:49.241197835 +0000 UTC m=+40.440664859 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle") pod "router-default-549c4cff84-xqpgj" (UID: "80edfd93-3c7a-4356-9c21-6b1376df14fb") : configmap references non-existent config key: service-ca.crt Jul 12 18:25:48.741338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741277 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23bc3496-ee12-455a-8e8a-6101006dbcb7-config\") pod \"console-operator-7866497494-9nvfk\" (UID: \"23bc3496-ee12-455a-8e8a-6101006dbcb7\") " pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:48.741338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741312 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.741490 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741337 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23bc3496-ee12-455a-8e8a-6101006dbcb7-serving-cert\") pod \"console-operator-7866497494-9nvfk\" (UID: \"23bc3496-ee12-455a-8e8a-6101006dbcb7\") " pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:48.741490 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741397 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/429df8ad-28e9-487c-a2d9-d66a44615bc4-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-pm7l6\" (UID: \"429df8ad-28e9-487c-a2d9-d66a44615bc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" Jul 12 18:25:48.741490 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741446 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a33fa508-623e-4693-88b0-51eb8cc79715-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:25:48.741490 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741475 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-tmp\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.741690 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741543 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.741690 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741586 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-stats-auth\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:48.741690 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741612 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:48.741690 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741641 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-snapshots\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.741690 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741670 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-default-certificate\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:48.741906 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741698 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6p27\" (UniqueName: \"kubernetes.io/projected/429df8ad-28e9-487c-a2d9-d66a44615bc4-kube-api-access-b6p27\") pod \"kube-storage-version-migrator-operator-799cb7f958-pm7l6\" (UID: \"429df8ad-28e9-487c-a2d9-d66a44615bc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" Jul 12 18:25:48.741906 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741770 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkqhv\" (UniqueName: \"kubernetes.io/projected/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-kube-api-access-pkqhv\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.741906 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:48.741795 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 12 18:25:48.741906 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:48.741859 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs podName:80edfd93-3c7a-4356-9c21-6b1376df14fb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:49.241842899 +0000 UTC m=+40.441309923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs") pod "router-default-549c4cff84-xqpgj" (UID: "80edfd93-3c7a-4356-9c21-6b1376df14fb") : secret "router-metrics-certs-default" not found Jul 12 18:25:48.742122 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741908 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47b2x\" (UniqueName: \"kubernetes.io/projected/a33fa508-623e-4693-88b0-51eb8cc79715-kube-api-access-47b2x\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:25:48.742122 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741921 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.742122 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.741964 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plgkb\" (UniqueName: \"kubernetes.io/projected/78c7b129-3082-4dc2-a271-d98c4ae27fbc-kube-api-access-plgkb\") pod \"network-check-source-869dcfcdcd-8sjbq\" (UID: \"78c7b129-3082-4dc2-a271-d98c4ae27fbc\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8sjbq" Jul 12 18:25:48.742122 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.742006 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-serving-cert\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.742122 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.742032 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:25:48.742122 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.742079 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smkhw\" (UniqueName: \"kubernetes.io/projected/23bc3496-ee12-455a-8e8a-6101006dbcb7-kube-api-access-smkhw\") pod \"console-operator-7866497494-9nvfk\" (UID: \"23bc3496-ee12-455a-8e8a-6101006dbcb7\") " pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:48.742122 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.742103 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/429df8ad-28e9-487c-a2d9-d66a44615bc4-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-pm7l6\" (UID: \"429df8ad-28e9-487c-a2d9-d66a44615bc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" Jul 12 18:25:48.742441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.742129 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23bc3496-ee12-455a-8e8a-6101006dbcb7-trusted-ca\") pod \"console-operator-7866497494-9nvfk\" (UID: \"23bc3496-ee12-455a-8e8a-6101006dbcb7\") " pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:48.742441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.742386 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.742738 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.742716 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-snapshots\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.744492 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.744472 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-serving-cert\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.744650 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.744631 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-default-certificate\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:48.744686 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.744630 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-stats-auth\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:48.752229 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.752202 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkqhv\" (UniqueName: \"kubernetes.io/projected/b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2-kube-api-access-pkqhv\") pod \"insights-operator-68d9dd94cf-96sxt\" (UID: \"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2\") " pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.752337 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.752202 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrn7d\" (UniqueName: \"kubernetes.io/projected/80edfd93-3c7a-4356-9c21-6b1376df14fb-kube-api-access-nrn7d\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:48.785401 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.785370 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" Jul 12 18:25:48.793235 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.793205 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" Jul 12 18:25:48.836821 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.836790 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9cft2"] Jul 12 18:25:48.839918 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:48.839881 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca704d74_c4dc_4547_b226_2f910224db6c.slice/crio-ea81ab2fcd40fbcf86088ead5dabea1977f3a277a0bccb0a42848e2bc98a5048 WatchSource:0}: Error finding container ea81ab2fcd40fbcf86088ead5dabea1977f3a277a0bccb0a42848e2bc98a5048: Status 404 returned error can't find the container with id ea81ab2fcd40fbcf86088ead5dabea1977f3a277a0bccb0a42848e2bc98a5048 Jul 12 18:25:48.842547 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.842513 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6p27\" (UniqueName: \"kubernetes.io/projected/429df8ad-28e9-487c-a2d9-d66a44615bc4-kube-api-access-b6p27\") pod \"kube-storage-version-migrator-operator-799cb7f958-pm7l6\" (UID: \"429df8ad-28e9-487c-a2d9-d66a44615bc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" Jul 12 18:25:48.842664 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.842571 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47b2x\" (UniqueName: \"kubernetes.io/projected/a33fa508-623e-4693-88b0-51eb8cc79715-kube-api-access-47b2x\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:25:48.842771 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.842742 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plgkb\" (UniqueName: \"kubernetes.io/projected/78c7b129-3082-4dc2-a271-d98c4ae27fbc-kube-api-access-plgkb\") pod \"network-check-source-869dcfcdcd-8sjbq\" (UID: \"78c7b129-3082-4dc2-a271-d98c4ae27fbc\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8sjbq" Jul 12 18:25:48.842972 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:48.842955 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 18:25:48.843047 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:48.843004 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls podName:a33fa508-623e-4693-88b0-51eb8cc79715 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:49.342989547 +0000 UTC m=+40.542456556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-642k2" (UID: "a33fa508-623e-4693-88b0-51eb8cc79715") : secret "cluster-monitoring-operator-tls" not found Jul 12 18:25:48.843142 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.843114 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:25:48.843249 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.843185 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smkhw\" (UniqueName: \"kubernetes.io/projected/23bc3496-ee12-455a-8e8a-6101006dbcb7-kube-api-access-smkhw\") pod \"console-operator-7866497494-9nvfk\" (UID: \"23bc3496-ee12-455a-8e8a-6101006dbcb7\") " pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:48.843249 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.843212 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/429df8ad-28e9-487c-a2d9-d66a44615bc4-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-pm7l6\" (UID: \"429df8ad-28e9-487c-a2d9-d66a44615bc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" Jul 12 18:25:48.843249 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.843240 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23bc3496-ee12-455a-8e8a-6101006dbcb7-trusted-ca\") pod \"console-operator-7866497494-9nvfk\" (UID: \"23bc3496-ee12-455a-8e8a-6101006dbcb7\") " pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:48.843354 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.843321 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23bc3496-ee12-455a-8e8a-6101006dbcb7-config\") pod \"console-operator-7866497494-9nvfk\" (UID: \"23bc3496-ee12-455a-8e8a-6101006dbcb7\") " pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:48.843444 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.843352 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23bc3496-ee12-455a-8e8a-6101006dbcb7-serving-cert\") pod \"console-operator-7866497494-9nvfk\" (UID: \"23bc3496-ee12-455a-8e8a-6101006dbcb7\") " pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:48.843444 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.843390 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/429df8ad-28e9-487c-a2d9-d66a44615bc4-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-pm7l6\" (UID: \"429df8ad-28e9-487c-a2d9-d66a44615bc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" Jul 12 18:25:48.843537 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.843446 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a33fa508-623e-4693-88b0-51eb8cc79715-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:25:48.844126 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.844079 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/429df8ad-28e9-487c-a2d9-d66a44615bc4-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-pm7l6\" (UID: \"429df8ad-28e9-487c-a2d9-d66a44615bc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" Jul 12 18:25:48.844219 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.844194 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23bc3496-ee12-455a-8e8a-6101006dbcb7-trusted-ca\") pod \"console-operator-7866497494-9nvfk\" (UID: \"23bc3496-ee12-455a-8e8a-6101006dbcb7\") " pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:48.844219 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.844209 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23bc3496-ee12-455a-8e8a-6101006dbcb7-config\") pod \"console-operator-7866497494-9nvfk\" (UID: \"23bc3496-ee12-455a-8e8a-6101006dbcb7\") " pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:48.844358 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.844335 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a33fa508-623e-4693-88b0-51eb8cc79715-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:25:48.846213 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.846188 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23bc3496-ee12-455a-8e8a-6101006dbcb7-serving-cert\") pod \"console-operator-7866497494-9nvfk\" (UID: \"23bc3496-ee12-455a-8e8a-6101006dbcb7\") " pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:48.846596 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.846574 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/429df8ad-28e9-487c-a2d9-d66a44615bc4-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-pm7l6\" (UID: \"429df8ad-28e9-487c-a2d9-d66a44615bc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" Jul 12 18:25:48.853797 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.853533 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plgkb\" (UniqueName: \"kubernetes.io/projected/78c7b129-3082-4dc2-a271-d98c4ae27fbc-kube-api-access-plgkb\") pod \"network-check-source-869dcfcdcd-8sjbq\" (UID: \"78c7b129-3082-4dc2-a271-d98c4ae27fbc\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8sjbq" Jul 12 18:25:48.853797 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.853618 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6p27\" (UniqueName: \"kubernetes.io/projected/429df8ad-28e9-487c-a2d9-d66a44615bc4-kube-api-access-b6p27\") pod \"kube-storage-version-migrator-operator-799cb7f958-pm7l6\" (UID: \"429df8ad-28e9-487c-a2d9-d66a44615bc4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" Jul 12 18:25:48.854247 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.854168 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47b2x\" (UniqueName: \"kubernetes.io/projected/a33fa508-623e-4693-88b0-51eb8cc79715-kube-api-access-47b2x\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:25:48.854437 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.854252 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smkhw\" (UniqueName: \"kubernetes.io/projected/23bc3496-ee12-455a-8e8a-6101006dbcb7-kube-api-access-smkhw\") pod \"console-operator-7866497494-9nvfk\" (UID: \"23bc3496-ee12-455a-8e8a-6101006dbcb7\") " pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:48.906242 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.906212 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8sjbq" Jul 12 18:25:48.944648 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.942441 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k"] Jul 12 18:25:48.945814 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.945741 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:48.951319 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.951294 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-96sxt"] Jul 12 18:25:48.954437 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:48.954402 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e11c33_f3a3_4d9d_8d5b_1cd95defcea2.slice/crio-58176eeb1957a8a007e3c654e93dcd56cda085986a6e1dadb407b2f6f6954810 WatchSource:0}: Error finding container 58176eeb1957a8a007e3c654e93dcd56cda085986a6e1dadb407b2f6f6954810: Status 404 returned error can't find the container with id 58176eeb1957a8a007e3c654e93dcd56cda085986a6e1dadb407b2f6f6954810 Jul 12 18:25:48.965141 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:48.965109 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" Jul 12 18:25:49.084897 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.084634 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-8sjbq"] Jul 12 18:25:49.088842 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:49.088815 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c7b129_3082_4dc2_a271_d98c4ae27fbc.slice/crio-d82287950ff818a90ac92e689c7365a114f50bb2950842a9d16641fc6f302e64 WatchSource:0}: Error finding container d82287950ff818a90ac92e689c7365a114f50bb2950842a9d16641fc6f302e64: Status 404 returned error can't find the container with id d82287950ff818a90ac92e689c7365a114f50bb2950842a9d16641fc6f302e64 Jul 12 18:25:49.113180 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.113149 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-9nvfk"] Jul 12 18:25:49.115885 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:49.115856 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23bc3496_ee12_455a_8e8a_6101006dbcb7.slice/crio-17aa97e7d1cfea69180e286d75b7fc00cddedb1d3726927fa6a4ad5c672c7f8f WatchSource:0}: Error finding container 17aa97e7d1cfea69180e286d75b7fc00cddedb1d3726927fa6a4ad5c672c7f8f: Status 404 returned error can't find the container with id 17aa97e7d1cfea69180e286d75b7fc00cddedb1d3726927fa6a4ad5c672c7f8f Jul 12 18:25:49.140494 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.140415 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6"] Jul 12 18:25:49.143593 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:25:49.143564 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod429df8ad_28e9_487c_a2d9_d66a44615bc4.slice/crio-c1622054ab5cfaba82e5a720eaf7c1bd2961b9702c33799bad3c483527d5b00b WatchSource:0}: Error finding container c1622054ab5cfaba82e5a720eaf7c1bd2961b9702c33799bad3c483527d5b00b: Status 404 returned error can't find the container with id c1622054ab5cfaba82e5a720eaf7c1bd2961b9702c33799bad3c483527d5b00b Jul 12 18:25:49.149324 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.149302 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-qrltf\" (UID: \"b5d716f4-9338-4351-a5fc-46b1513626d0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" Jul 12 18:25:49.149445 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.149433 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-kzl7d\" (UID: \"d1e3d19c-3412-49a9-a51f-4658c02209bd\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" Jul 12 18:25:49.149500 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:49.149445 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 18:25:49.149542 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:49.149499 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls podName:b5d716f4-9338-4351-a5fc-46b1513626d0 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:50.149484703 +0000 UTC m=+41.348951714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls") pod "cluster-samples-operator-65db858894-qrltf" (UID: "b5d716f4-9338-4351-a5fc-46b1513626d0") : secret "samples-operator-tls" not found Jul 12 18:25:49.149542 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:49.149515 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 18:25:49.149629 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:49.149563 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert podName:d1e3d19c-3412-49a9-a51f-4658c02209bd nodeName:}" failed. No retries permitted until 2026-07-12 18:25:50.149543048 +0000 UTC m=+41.349010055 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-kzl7d" (UID: "d1e3d19c-3412-49a9-a51f-4658c02209bd") : secret "networking-console-plugin-cert" not found Jul 12 18:25:49.250086 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.250049 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:49.250255 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.250161 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:49.250255 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:49.250221 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle podName:80edfd93-3c7a-4356-9c21-6b1376df14fb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:50.250202133 +0000 UTC m=+41.449669144 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle") pod "router-default-549c4cff84-xqpgj" (UID: "80edfd93-3c7a-4356-9c21-6b1376df14fb") : configmap references non-existent config key: service-ca.crt Jul 12 18:25:49.250329 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:49.250258 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 12 18:25:49.250329 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:49.250303 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs podName:80edfd93-3c7a-4356-9c21-6b1376df14fb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:50.250288356 +0000 UTC m=+41.449755365 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs") pod "router-default-549c4cff84-xqpgj" (UID: "80edfd93-3c7a-4356-9c21-6b1376df14fb") : secret "router-metrics-certs-default" not found Jul 12 18:25:49.350678 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.350642 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:25:49.351323 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:49.350804 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 18:25:49.351323 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:49.350878 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls podName:a33fa508-623e-4693-88b0-51eb8cc79715 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:50.350863092 +0000 UTC m=+41.550330100 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-642k2" (UID: "a33fa508-623e-4693-88b0-51eb8cc79715") : secret "cluster-monitoring-operator-tls" not found Jul 12 18:25:49.520136 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.519910 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-9nvfk" event={"ID":"23bc3496-ee12-455a-8e8a-6101006dbcb7","Type":"ContainerStarted","Data":"17aa97e7d1cfea69180e286d75b7fc00cddedb1d3726927fa6a4ad5c672c7f8f"} Jul 12 18:25:49.521045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.521009 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8sjbq" event={"ID":"78c7b129-3082-4dc2-a271-d98c4ae27fbc","Type":"ContainerStarted","Data":"d82287950ff818a90ac92e689c7365a114f50bb2950842a9d16641fc6f302e64"} Jul 12 18:25:49.524414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.524386 2585 generic.go:356] "Generic (PLEG): container finished" podID="dd93dddb-eaaa-470d-84a4-17c4f61493e8" containerID="e49f017ef6368e29016006fee334530669bd84dc066db4817da10996e3c3d11c" exitCode=0 Jul 12 18:25:49.524530 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.524457 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llfzt" event={"ID":"dd93dddb-eaaa-470d-84a4-17c4f61493e8","Type":"ContainerDied","Data":"e49f017ef6368e29016006fee334530669bd84dc066db4817da10996e3c3d11c"} Jul 12 18:25:49.525788 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.525761 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" event={"ID":"429df8ad-28e9-487c-a2d9-d66a44615bc4","Type":"ContainerStarted","Data":"c1622054ab5cfaba82e5a720eaf7c1bd2961b9702c33799bad3c483527d5b00b"} Jul 12 18:25:49.526884 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.526861 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" event={"ID":"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2","Type":"ContainerStarted","Data":"58176eeb1957a8a007e3c654e93dcd56cda085986a6e1dadb407b2f6f6954810"} Jul 12 18:25:49.528353 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.528327 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" event={"ID":"621b603b-b0fd-4925-bbc6-015e3fd4d097","Type":"ContainerStarted","Data":"adcbe99a7a551f121df057d0c20a13bb8f3ed7b25159a00c723094ae824acef4"} Jul 12 18:25:49.529698 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:49.529675 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9cft2" event={"ID":"ca704d74-c4dc-4547-b226-2f910224db6c","Type":"ContainerStarted","Data":"ea81ab2fcd40fbcf86088ead5dabea1977f3a277a0bccb0a42848e2bc98a5048"} Jul 12 18:25:50.160096 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:50.159338 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-kzl7d\" (UID: \"d1e3d19c-3412-49a9-a51f-4658c02209bd\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" Jul 12 18:25:50.160096 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:50.159398 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-qrltf\" (UID: \"b5d716f4-9338-4351-a5fc-46b1513626d0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" Jul 12 18:25:50.160096 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:50.159566 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 18:25:50.160096 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:50.159630 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls podName:b5d716f4-9338-4351-a5fc-46b1513626d0 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:52.159613088 +0000 UTC m=+43.359080100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls") pod "cluster-samples-operator-65db858894-qrltf" (UID: "b5d716f4-9338-4351-a5fc-46b1513626d0") : secret "samples-operator-tls" not found Jul 12 18:25:50.160096 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:50.159692 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 18:25:50.160096 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:50.159727 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert podName:d1e3d19c-3412-49a9-a51f-4658c02209bd nodeName:}" failed. No retries permitted until 2026-07-12 18:25:52.159717107 +0000 UTC m=+43.359184119 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-kzl7d" (UID: "d1e3d19c-3412-49a9-a51f-4658c02209bd") : secret "networking-console-plugin-cert" not found Jul 12 18:25:50.261149 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:50.260328 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:50.261149 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:50.260479 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:50.261149 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:50.260611 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 12 18:25:50.261149 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:50.260672 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs podName:80edfd93-3c7a-4356-9c21-6b1376df14fb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:52.260654273 +0000 UTC m=+43.460121287 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs") pod "router-default-549c4cff84-xqpgj" (UID: "80edfd93-3c7a-4356-9c21-6b1376df14fb") : secret "router-metrics-certs-default" not found Jul 12 18:25:50.261149 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:50.261098 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle podName:80edfd93-3c7a-4356-9c21-6b1376df14fb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:52.261080535 +0000 UTC m=+43.460547547 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle") pod "router-default-549c4cff84-xqpgj" (UID: "80edfd93-3c7a-4356-9c21-6b1376df14fb") : configmap references non-existent config key: service-ca.crt Jul 12 18:25:50.362277 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:50.361525 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:25:50.362277 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:50.361824 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 18:25:50.362277 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:50.361901 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls podName:a33fa508-623e-4693-88b0-51eb8cc79715 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:52.361871946 +0000 UTC m=+43.561338968 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-642k2" (UID: "a33fa508-623e-4693-88b0-51eb8cc79715") : secret "cluster-monitoring-operator-tls" not found Jul 12 18:25:50.540378 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:50.540258 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-llfzt" event={"ID":"dd93dddb-eaaa-470d-84a4-17c4f61493e8","Type":"ContainerStarted","Data":"85cfde3d9cb8798a195002db34cb5b855b40dc6afbfb160d3535e2d20515598d"} Jul 12 18:25:50.565355 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:50.565257 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-llfzt" podStartSLOduration=8.092967595 podStartE2EDuration="40.56523506s" totalImagesPullingTime="32.472267465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:50.562460882 +0000 UTC m=+41.761927914" watchObservedRunningTime="2026-07-12 18:25:50.56523506 +0000 UTC m=+41.764702091" Jul 12 18:25:51.545452 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:51.545406 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9cft2" event={"ID":"ca704d74-c4dc-4547-b226-2f910224db6c","Type":"ContainerStarted","Data":"41e83f8364ace09b75865fb8d4c30621eb7e4b0abe3a220f43193d173242c4ee"} Jul 12 18:25:51.562831 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:51.562766 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-9cft2" podStartSLOduration=1.805389905 podStartE2EDuration="3.562745756s" totalImagesPullingTime="1.757355851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:51.562541271 +0000 UTC m=+42.762008301" watchObservedRunningTime="2026-07-12 18:25:51.562745756 +0000 UTC m=+42.762212788" Jul 12 18:25:52.187274 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:52.187230 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-kzl7d\" (UID: \"d1e3d19c-3412-49a9-a51f-4658c02209bd\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" Jul 12 18:25:52.187486 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:52.187288 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-qrltf\" (UID: \"b5d716f4-9338-4351-a5fc-46b1513626d0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" Jul 12 18:25:52.187486 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:52.187389 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 18:25:52.187486 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:52.187461 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert podName:d1e3d19c-3412-49a9-a51f-4658c02209bd nodeName:}" failed. No retries permitted until 2026-07-12 18:25:56.187444381 +0000 UTC m=+47.386911389 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-kzl7d" (UID: "d1e3d19c-3412-49a9-a51f-4658c02209bd") : secret "networking-console-plugin-cert" not found Jul 12 18:25:52.187486 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:52.187459 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 18:25:52.187486 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:52.187488 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls podName:b5d716f4-9338-4351-a5fc-46b1513626d0 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:56.187481686 +0000 UTC m=+47.386948694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls") pod "cluster-samples-operator-65db858894-qrltf" (UID: "b5d716f4-9338-4351-a5fc-46b1513626d0") : secret "samples-operator-tls" not found Jul 12 18:25:52.288697 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:52.288657 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:52.288879 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:52.288744 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:52.288879 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:52.288813 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 12 18:25:52.288879 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:52.288852 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle podName:80edfd93-3c7a-4356-9c21-6b1376df14fb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:56.288838666 +0000 UTC m=+47.488305679 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle") pod "router-default-549c4cff84-xqpgj" (UID: "80edfd93-3c7a-4356-9c21-6b1376df14fb") : configmap references non-existent config key: service-ca.crt Jul 12 18:25:52.288879 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:52.288867 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs podName:80edfd93-3c7a-4356-9c21-6b1376df14fb nodeName:}" failed. No retries permitted until 2026-07-12 18:25:56.288860713 +0000 UTC m=+47.488327720 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs") pod "router-default-549c4cff84-xqpgj" (UID: "80edfd93-3c7a-4356-9c21-6b1376df14fb") : secret "router-metrics-certs-default" not found Jul 12 18:25:52.389785 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:52.389743 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:25:52.389967 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:52.389863 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:25:52.389967 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:52.389890 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 18:25:52.389967 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:52.389911 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bd449f9f-lf78m: secret "image-registry-tls" not found Jul 12 18:25:52.389967 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:52.389946 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:25:52.390148 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:52.389975 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls podName:cfdb85db-878a-43b4-af66-ebe34aea186a nodeName:}" failed. No retries permitted until 2026-07-12 18:26:00.38996043 +0000 UTC m=+51.589427437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls") pod "image-registry-67bd449f9f-lf78m" (UID: "cfdb85db-878a-43b4-af66-ebe34aea186a") : secret "image-registry-tls" not found Jul 12 18:25:52.390148 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:52.390025 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 18:25:52.390148 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:52.390080 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls podName:a33fa508-623e-4693-88b0-51eb8cc79715 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:56.390063387 +0000 UTC m=+47.589530411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-642k2" (UID: "a33fa508-623e-4693-88b0-51eb8cc79715") : secret "cluster-monitoring-operator-tls" not found Jul 12 18:25:52.390148 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:52.390026 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 18:25:52.390148 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:52.390121 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls podName:cb62dd7e-0d0f-4128-87dd-3158aa45a396 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:00.390111768 +0000 UTC m=+51.589578777 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls") pod "dns-default-lphwv" (UID: "cb62dd7e-0d0f-4128-87dd-3158aa45a396") : secret "dns-default-metrics-tls" not found Jul 12 18:25:55.572622 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:55.571971 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" event={"ID":"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2","Type":"ContainerStarted","Data":"d12d28703776285c9a422a8190c31852b11fa6df28a44280e9f1a371c74e1e83"} Jul 12 18:25:55.574954 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:55.574693 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" event={"ID":"621b603b-b0fd-4925-bbc6-015e3fd4d097","Type":"ContainerStarted","Data":"f513f071e9ce88b3308c6ded41a7b0860c7241e7e60f272dd979c8d1962fc826"} Jul 12 18:25:55.576559 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:55.576538 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/0.log" Jul 12 18:25:55.576661 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:55.576577 2585 generic.go:356] "Generic (PLEG): container finished" podID="23bc3496-ee12-455a-8e8a-6101006dbcb7" containerID="2a5ee09af776608b3c186bc96764e3f502eea1e76429be158af26a46b7448e21" exitCode=255 Jul 12 18:25:55.576874 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:55.576847 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-9nvfk" event={"ID":"23bc3496-ee12-455a-8e8a-6101006dbcb7","Type":"ContainerDied","Data":"2a5ee09af776608b3c186bc96764e3f502eea1e76429be158af26a46b7448e21"} Jul 12 18:25:55.576977 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:55.576906 2585 scope.go:117] "RemoveContainer" containerID="2a5ee09af776608b3c186bc96764e3f502eea1e76429be158af26a46b7448e21" Jul 12 18:25:55.579876 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:55.579825 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8sjbq" event={"ID":"78c7b129-3082-4dc2-a271-d98c4ae27fbc","Type":"ContainerStarted","Data":"16bcee0d2a0b3dee1fc0282ea8405f496900a046606750669e8cfa7059b9d0ca"} Jul 12 18:25:55.580804 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:55.580470 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" event={"ID":"429df8ad-28e9-487c-a2d9-d66a44615bc4","Type":"ContainerStarted","Data":"9331c9ff47bcbec3f722a5f0c9fcc416de64f89f5f2baa2aecea4d85b21af390"} Jul 12 18:25:55.590147 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:55.589473 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" podStartSLOduration=1.235732007 podStartE2EDuration="7.58945295s" totalImagesPullingTime="6.353720943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:55.588843523 +0000 UTC m=+46.788310553" watchObservedRunningTime="2026-07-12 18:25:55.58945295 +0000 UTC m=+46.788919982" Jul 12 18:25:55.605900 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:55.605592 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" podStartSLOduration=1.242820201 podStartE2EDuration="7.605568573s" totalImagesPullingTime="6.362748372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:55.603000831 +0000 UTC m=+46.802467863" watchObservedRunningTime="2026-07-12 18:25:55.605568573 +0000 UTC m=+46.805035585" Jul 12 18:25:55.642475 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:55.642282 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" podStartSLOduration=1.477317222 podStartE2EDuration="7.642259635s" totalImagesPullingTime="6.164942413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:55.640511454 +0000 UTC m=+46.839978497" watchObservedRunningTime="2026-07-12 18:25:55.642259635 +0000 UTC m=+46.841726674" Jul 12 18:25:56.225514 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.225465 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-kzl7d\" (UID: \"d1e3d19c-3412-49a9-a51f-4658c02209bd\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" Jul 12 18:25:56.225738 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.225526 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-qrltf\" (UID: \"b5d716f4-9338-4351-a5fc-46b1513626d0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" Jul 12 18:25:56.225738 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:56.225641 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 18:25:56.225738 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:56.225693 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 18:25:56.225738 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:56.225720 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert podName:d1e3d19c-3412-49a9-a51f-4658c02209bd nodeName:}" failed. No retries permitted until 2026-07-12 18:26:04.225699568 +0000 UTC m=+55.425166591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-kzl7d" (UID: "d1e3d19c-3412-49a9-a51f-4658c02209bd") : secret "networking-console-plugin-cert" not found Jul 12 18:25:56.225983 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:56.225757 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls podName:b5d716f4-9338-4351-a5fc-46b1513626d0 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:04.225739788 +0000 UTC m=+55.425206809 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls") pod "cluster-samples-operator-65db858894-qrltf" (UID: "b5d716f4-9338-4351-a5fc-46b1513626d0") : secret "samples-operator-tls" not found Jul 12 18:25:56.326982 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.326918 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:56.327161 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.327090 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:25:56.327161 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:56.327122 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle podName:80edfd93-3c7a-4356-9c21-6b1376df14fb nodeName:}" failed. No retries permitted until 2026-07-12 18:26:04.327100021 +0000 UTC m=+55.526567042 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle") pod "router-default-549c4cff84-xqpgj" (UID: "80edfd93-3c7a-4356-9c21-6b1376df14fb") : configmap references non-existent config key: service-ca.crt Jul 12 18:25:56.327256 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:56.327187 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 12 18:25:56.327256 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:56.327247 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs podName:80edfd93-3c7a-4356-9c21-6b1376df14fb nodeName:}" failed. No retries permitted until 2026-07-12 18:26:04.327221804 +0000 UTC m=+55.526688817 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs") pod "router-default-549c4cff84-xqpgj" (UID: "80edfd93-3c7a-4356-9c21-6b1376df14fb") : secret "router-metrics-certs-default" not found Jul 12 18:25:56.428088 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.428023 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:25:56.428283 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:56.428192 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 18:25:56.428283 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:56.428274 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls podName:a33fa508-623e-4693-88b0-51eb8cc79715 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:04.428253567 +0000 UTC m=+55.627720591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-642k2" (UID: "a33fa508-623e-4693-88b0-51eb8cc79715") : secret "cluster-monitoring-operator-tls" not found Jul 12 18:25:56.587177 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.587146 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:25:56.587870 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.587854 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/0.log" Jul 12 18:25:56.587964 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.587900 2585 generic.go:356] "Generic (PLEG): container finished" podID="23bc3496-ee12-455a-8e8a-6101006dbcb7" containerID="69c331abe5c596b39da01a6ae7479c07e76fb448aee07c840c82dbb2d39deccb" exitCode=255 Jul 12 18:25:56.588061 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.588037 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-9nvfk" event={"ID":"23bc3496-ee12-455a-8e8a-6101006dbcb7","Type":"ContainerDied","Data":"69c331abe5c596b39da01a6ae7479c07e76fb448aee07c840c82dbb2d39deccb"} Jul 12 18:25:56.588115 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.588105 2585 scope.go:117] "RemoveContainer" containerID="2a5ee09af776608b3c186bc96764e3f502eea1e76429be158af26a46b7448e21" Jul 12 18:25:56.588431 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.588417 2585 scope.go:117] "RemoveContainer" containerID="69c331abe5c596b39da01a6ae7479c07e76fb448aee07c840c82dbb2d39deccb" Jul 12 18:25:56.588619 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:56.588600 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-9nvfk_openshift-console-operator(23bc3496-ee12-455a-8e8a-6101006dbcb7)\"" pod="openshift-console-operator/console-operator-7866497494-9nvfk" podUID="23bc3496-ee12-455a-8e8a-6101006dbcb7" Jul 12 18:25:56.606682 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.606629 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8sjbq" podStartSLOduration=2.37788481 podStartE2EDuration="8.606612571s" totalImagesPullingTime="6.228727761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:25:55.655265503 +0000 UTC m=+46.854732529" watchObservedRunningTime="2026-07-12 18:25:56.606612571 +0000 UTC m=+47.806079601" Jul 12 18:25:56.822749 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.822704 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r5d4d"] Jul 12 18:25:56.845588 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.845476 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r5d4d"] Jul 12 18:25:56.845792 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.845618 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:56.850275 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.849770 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 12 18:25:56.850275 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.849781 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4qrm6\"" Jul 12 18:25:56.850275 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.850117 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 12 18:25:56.934202 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.934166 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwr95\" (UniqueName: \"kubernetes.io/projected/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-api-access-fwr95\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:56.934376 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.934338 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e98e68e3-3c5c-45b0-9817-a05d41b96502-crio-socket\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:56.934428 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.934374 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:56.934428 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.934416 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e98e68e3-3c5c-45b0-9817-a05d41b96502-data-volume\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:56.934536 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:56.934518 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:57.035376 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:57.035330 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e98e68e3-3c5c-45b0-9817-a05d41b96502-crio-socket\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:57.035560 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:57.035385 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:57.035560 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:57.035424 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e98e68e3-3c5c-45b0-9817-a05d41b96502-data-volume\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:57.035560 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:57.035486 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:57.035710 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:57.035558 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 18:25:57.035710 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:57.035572 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwr95\" (UniqueName: \"kubernetes.io/projected/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-api-access-fwr95\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:57.035710 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:57.035620 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls podName:e98e68e3-3c5c-45b0-9817-a05d41b96502 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:57.535606526 +0000 UTC m=+48.735073534 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r5d4d" (UID: "e98e68e3-3c5c-45b0-9817-a05d41b96502") : secret "insights-runtime-extractor-tls" not found Jul 12 18:25:57.035710 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:57.035676 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 18:25:57.035914 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:57.035745 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm podName:e98e68e3-3c5c-45b0-9817-a05d41b96502 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:57.535726767 +0000 UTC m=+48.735193788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r5d4d" (UID: "e98e68e3-3c5c-45b0-9817-a05d41b96502") : configmap "kube-rbac-proxy" not found Jul 12 18:25:57.035914 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:57.035598 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e98e68e3-3c5c-45b0-9817-a05d41b96502-crio-socket\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:57.035914 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:57.035823 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e98e68e3-3c5c-45b0-9817-a05d41b96502-data-volume\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:57.046302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:57.046231 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwr95\" (UniqueName: \"kubernetes.io/projected/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-api-access-fwr95\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:57.538311 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:57.538267 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:57.538516 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:57.538384 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 18:25:57.538516 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:57.538410 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:57.538516 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:57.538459 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm podName:e98e68e3-3c5c-45b0-9817-a05d41b96502 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:58.538443867 +0000 UTC m=+49.737910875 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r5d4d" (UID: "e98e68e3-3c5c-45b0-9817-a05d41b96502") : configmap "kube-rbac-proxy" not found Jul 12 18:25:57.538634 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:57.538533 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 18:25:57.538634 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:57.538577 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls podName:e98e68e3-3c5c-45b0-9817-a05d41b96502 nodeName:}" failed. No retries permitted until 2026-07-12 18:25:58.538565829 +0000 UTC m=+49.738032837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r5d4d" (UID: "e98e68e3-3c5c-45b0-9817-a05d41b96502") : secret "insights-runtime-extractor-tls" not found Jul 12 18:25:57.596124 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:57.596098 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:25:57.596504 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:57.596413 2585 scope.go:117] "RemoveContainer" containerID="69c331abe5c596b39da01a6ae7479c07e76fb448aee07c840c82dbb2d39deccb" Jul 12 18:25:57.596592 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:57.596573 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-9nvfk_openshift-console-operator(23bc3496-ee12-455a-8e8a-6101006dbcb7)\"" pod="openshift-console-operator/console-operator-7866497494-9nvfk" podUID="23bc3496-ee12-455a-8e8a-6101006dbcb7" Jul 12 18:25:58.408832 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:58.408806 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x7b84_fa50005b-ad03-474a-a648-f9fde6a932a6/dns-node-resolver/0.log" Jul 12 18:25:58.547238 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:58.547196 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:58.547389 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:58.547267 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:25:58.547389 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:58.547333 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 18:25:58.547389 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:58.547354 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 18:25:58.547504 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:58.547403 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm podName:e98e68e3-3c5c-45b0-9817-a05d41b96502 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:00.547387441 +0000 UTC m=+51.746854449 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r5d4d" (UID: "e98e68e3-3c5c-45b0-9817-a05d41b96502") : configmap "kube-rbac-proxy" not found Jul 12 18:25:58.547504 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:58.547417 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls podName:e98e68e3-3c5c-45b0-9817-a05d41b96502 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:00.547411637 +0000 UTC m=+51.746878646 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r5d4d" (UID: "e98e68e3-3c5c-45b0-9817-a05d41b96502") : secret "insights-runtime-extractor-tls" not found Jul 12 18:25:58.946910 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:58.946871 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:58.947336 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:58.946992 2585 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:25:58.947379 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:58.947333 2585 scope.go:117] "RemoveContainer" containerID="69c331abe5c596b39da01a6ae7479c07e76fb448aee07c840c82dbb2d39deccb" Jul 12 18:25:58.947495 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:58.947476 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-9nvfk_openshift-console-operator(23bc3496-ee12-455a-8e8a-6101006dbcb7)\"" pod="openshift-console-operator/console-operator-7866497494-9nvfk" podUID="23bc3496-ee12-455a-8e8a-6101006dbcb7" Jul 12 18:25:59.209430 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:59.209349 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5dzg6_64c451f1-c731-40c5-a837-b515e65009e0/node-ca/0.log" Jul 12 18:25:59.602904 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:25:59.602872 2585 scope.go:117] "RemoveContainer" containerID="69c331abe5c596b39da01a6ae7479c07e76fb448aee07c840c82dbb2d39deccb" Jul 12 18:25:59.603124 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:25:59.603072 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-9nvfk_openshift-console-operator(23bc3496-ee12-455a-8e8a-6101006dbcb7)\"" pod="openshift-console-operator/console-operator-7866497494-9nvfk" podUID="23bc3496-ee12-455a-8e8a-6101006dbcb7" Jul 12 18:26:00.209478 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:00.209451 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l98hb_9aef0f72-416e-4c37-98b8-a68e30a65ce6/serve-healthcheck-canary/0.log" Jul 12 18:26:00.464843 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:00.464744 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:26:00.465028 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:00.464862 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:26:00.465028 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:00.464912 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 18:26:00.465028 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:00.465001 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls podName:cb62dd7e-0d0f-4128-87dd-3158aa45a396 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:16.464986197 +0000 UTC m=+67.664453205 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls") pod "dns-default-lphwv" (UID: "cb62dd7e-0d0f-4128-87dd-3158aa45a396") : secret "dns-default-metrics-tls" not found Jul 12 18:26:00.465028 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:00.465006 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 18:26:00.465028 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:00.465016 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67bd449f9f-lf78m: secret "image-registry-tls" not found Jul 12 18:26:00.465227 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:00.465055 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls podName:cfdb85db-878a-43b4-af66-ebe34aea186a nodeName:}" failed. No retries permitted until 2026-07-12 18:26:16.465043822 +0000 UTC m=+67.664510830 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls") pod "image-registry-67bd449f9f-lf78m" (UID: "cfdb85db-878a-43b4-af66-ebe34aea186a") : secret "image-registry-tls" not found Jul 12 18:26:00.565978 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:00.565913 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:26:00.566178 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:00.566007 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:26:00.566178 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:00.566081 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 18:26:00.566178 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:00.566163 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm podName:e98e68e3-3c5c-45b0-9817-a05d41b96502 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:04.566145752 +0000 UTC m=+55.765612760 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r5d4d" (UID: "e98e68e3-3c5c-45b0-9817-a05d41b96502") : configmap "kube-rbac-proxy" not found Jul 12 18:26:00.566350 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:00.566087 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 18:26:00.566350 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:00.566227 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls podName:e98e68e3-3c5c-45b0-9817-a05d41b96502 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:04.566214564 +0000 UTC m=+55.765681573 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r5d4d" (UID: "e98e68e3-3c5c-45b0-9817-a05d41b96502") : secret "insights-runtime-extractor-tls" not found Jul 12 18:26:04.300689 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:04.300644 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-kzl7d\" (UID: \"d1e3d19c-3412-49a9-a51f-4658c02209bd\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" Jul 12 18:26:04.300689 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:04.300695 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-qrltf\" (UID: \"b5d716f4-9338-4351-a5fc-46b1513626d0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" Jul 12 18:26:04.301331 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:04.300797 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 18:26:04.301331 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:04.300800 2585 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 12 18:26:04.301331 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:04.300871 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls podName:b5d716f4-9338-4351-a5fc-46b1513626d0 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:20.300853194 +0000 UTC m=+71.500320204 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls") pod "cluster-samples-operator-65db858894-qrltf" (UID: "b5d716f4-9338-4351-a5fc-46b1513626d0") : secret "samples-operator-tls" not found Jul 12 18:26:04.301331 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:04.300887 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert podName:d1e3d19c-3412-49a9-a51f-4658c02209bd nodeName:}" failed. No retries permitted until 2026-07-12 18:26:20.300879336 +0000 UTC m=+71.500346343 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-kzl7d" (UID: "d1e3d19c-3412-49a9-a51f-4658c02209bd") : secret "networking-console-plugin-cert" not found Jul 12 18:26:04.402056 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:04.402014 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:26:04.402224 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:04.402162 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:26:04.402224 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:04.402175 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle podName:80edfd93-3c7a-4356-9c21-6b1376df14fb nodeName:}" failed. No retries permitted until 2026-07-12 18:26:20.40215715 +0000 UTC m=+71.601624157 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle") pod "router-default-549c4cff84-xqpgj" (UID: "80edfd93-3c7a-4356-9c21-6b1376df14fb") : configmap references non-existent config key: service-ca.crt Jul 12 18:26:04.402356 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:04.402254 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 12 18:26:04.402356 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:04.402314 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs podName:80edfd93-3c7a-4356-9c21-6b1376df14fb nodeName:}" failed. No retries permitted until 2026-07-12 18:26:20.402298459 +0000 UTC m=+71.601765493 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs") pod "router-default-549c4cff84-xqpgj" (UID: "80edfd93-3c7a-4356-9c21-6b1376df14fb") : secret "router-metrics-certs-default" not found Jul 12 18:26:04.503236 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:04.503196 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:26:04.503392 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:04.503354 2585 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 18:26:04.503452 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:04.503425 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls podName:a33fa508-623e-4693-88b0-51eb8cc79715 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:20.503410583 +0000 UTC m=+71.702877590 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-642k2" (UID: "a33fa508-623e-4693-88b0-51eb8cc79715") : secret "cluster-monitoring-operator-tls" not found Jul 12 18:26:04.604194 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:04.604111 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:26:04.604353 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:04.604264 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:26:04.604400 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:04.604362 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 18:26:04.604433 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:04.604415 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls podName:e98e68e3-3c5c-45b0-9817-a05d41b96502 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:12.604402889 +0000 UTC m=+63.803869902 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r5d4d" (UID: "e98e68e3-3c5c-45b0-9817-a05d41b96502") : secret "insights-runtime-extractor-tls" not found Jul 12 18:26:04.604741 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:04.604720 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e98e68e3-3c5c-45b0-9817-a05d41b96502-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:26:11.498283 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:11.498255 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gswlt" Jul 12 18:26:12.299097 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:12.299057 2585 scope.go:117] "RemoveContainer" containerID="69c331abe5c596b39da01a6ae7479c07e76fb448aee07c840c82dbb2d39deccb" Jul 12 18:26:12.641249 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:12.641164 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:26:12.641249 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:12.641233 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-9nvfk" event={"ID":"23bc3496-ee12-455a-8e8a-6101006dbcb7","Type":"ContainerStarted","Data":"2af56064d059a73bcc285d14dc69ae138d984435d7b183c4098224dd7825386b"} Jul 12 18:26:12.641768 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:12.641499 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:26:12.658349 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:12.658290 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-9nvfk" podStartSLOduration=18.465863312 podStartE2EDuration="24.658268408s" totalImagesPullingTime="6.192405096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:12.65808233 +0000 UTC m=+63.857549360" watchObservedRunningTime="2026-07-12 18:26:12.658268408 +0000 UTC m=+63.857735437" Jul 12 18:26:12.676371 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:12.676343 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:26:12.678809 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:12.678772 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e98e68e3-3c5c-45b0-9817-a05d41b96502-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r5d4d\" (UID: \"e98e68e3-3c5c-45b0-9817-a05d41b96502\") " pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:26:12.759625 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:12.759597 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4qrm6\"" Jul 12 18:26:12.767562 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:12.767536 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r5d4d" Jul 12 18:26:12.899377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:12.899241 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r5d4d"] Jul 12 18:26:12.902566 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:26:12.902538 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode98e68e3_3c5c_45b0_9817_a05d41b96502.slice/crio-9e2f8b334e29733902610e0786add07c5653bbe0d24401e6751b0ddf1271f786 WatchSource:0}: Error finding container 9e2f8b334e29733902610e0786add07c5653bbe0d24401e6751b0ddf1271f786: Status 404 returned error can't find the container with id 9e2f8b334e29733902610e0786add07c5653bbe0d24401e6751b0ddf1271f786 Jul 12 18:26:12.952241 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:12.952216 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-9nvfk" Jul 12 18:26:13.648423 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:13.648334 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r5d4d" event={"ID":"e98e68e3-3c5c-45b0-9817-a05d41b96502","Type":"ContainerStarted","Data":"9e2f8b334e29733902610e0786add07c5653bbe0d24401e6751b0ddf1271f786"} Jul 12 18:26:15.003374 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:15.003340 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs\") pod \"network-metrics-daemon-rgkpj\" (UID: \"87644bbc-d561-4ca5-8b2b-38bf70f01efb\") " pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:26:15.006035 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:15.006015 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 18:26:15.016304 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:15.016281 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87644bbc-d561-4ca5-8b2b-38bf70f01efb-metrics-certs\") pod \"network-metrics-daemon-rgkpj\" (UID: \"87644bbc-d561-4ca5-8b2b-38bf70f01efb\") " pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:26:15.206158 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:15.206118 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz8h8\" (UniqueName: \"kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8\") pod \"network-check-target-8q6bw\" (UID: \"7abc23a1-1bde-4046-a60d-cf3e93e64f63\") " pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:26:15.206335 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:15.206236 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:26:15.208976 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:15.208951 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 18:26:15.209114 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:15.209051 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz8h8\" (UniqueName: \"kubernetes.io/projected/7abc23a1-1bde-4046-a60d-cf3e93e64f63-kube-api-access-wz8h8\") pod \"network-check-target-8q6bw\" (UID: \"7abc23a1-1bde-4046-a60d-cf3e93e64f63\") " pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:26:15.219574 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:15.219540 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ad0a79f-69ee-49e8-90eb-c27482619b69-original-pull-secret\") pod \"global-pull-secret-syncer-j5bpc\" (UID: \"6ad0a79f-69ee-49e8-90eb-c27482619b69\") " pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:26:15.310441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:15.310409 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gx6mz\"" Jul 12 18:26:15.318463 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:15.318416 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgkpj" Jul 12 18:26:15.411572 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:15.411534 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j5bpc" Jul 12 18:26:15.420896 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:15.420859 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p8prd\"" Jul 12 18:26:15.428846 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:15.428817 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:26:16.518011 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:16.517956 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:26:16.518407 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:16.518075 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:26:16.520645 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:16.520620 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb62dd7e-0d0f-4128-87dd-3158aa45a396-metrics-tls\") pod \"dns-default-lphwv\" (UID: \"cb62dd7e-0d0f-4128-87dd-3158aa45a396\") " pod="openshift-dns/dns-default-lphwv" Jul 12 18:26:16.521342 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:16.521314 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls\") pod \"image-registry-67bd449f9f-lf78m\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:26:16.687201 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:16.687164 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-twcn4\"" Jul 12 18:26:16.695253 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:16.695200 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:26:16.704321 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:16.704290 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7m82z\"" Jul 12 18:26:16.712303 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:16.712274 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lphwv" Jul 12 18:26:16.910723 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:16.910692 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j5bpc"] Jul 12 18:26:16.922127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:16.922051 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rgkpj"] Jul 12 18:26:16.945651 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:16.945626 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67bd449f9f-lf78m"] Jul 12 18:26:16.992866 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:26:16.992828 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad0a79f_69ee_49e8_90eb_c27482619b69.slice/crio-8e915ca20c2bfd56d62052a806509a23d6262c98b1cf744e6ebfe8002f901a4d WatchSource:0}: Error finding container 8e915ca20c2bfd56d62052a806509a23d6262c98b1cf744e6ebfe8002f901a4d: Status 404 returned error can't find the container with id 8e915ca20c2bfd56d62052a806509a23d6262c98b1cf744e6ebfe8002f901a4d Jul 12 18:26:16.993510 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:26:16.993481 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87644bbc_d561_4ca5_8b2b_38bf70f01efb.slice/crio-5db5d057f1d45473d74cdca18c62fa25bfbb8c45e193710e9cfdc6585c4d3b60 WatchSource:0}: Error finding container 5db5d057f1d45473d74cdca18c62fa25bfbb8c45e193710e9cfdc6585c4d3b60: Status 404 returned error can't find the container with id 5db5d057f1d45473d74cdca18c62fa25bfbb8c45e193710e9cfdc6585c4d3b60 Jul 12 18:26:16.994257 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:26:16.994234 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfdb85db_878a_43b4_af66_ebe34aea186a.slice/crio-dd688ceca132d007a07f6d9d89531f8d9821aadd185047eb621e88bad3d0f01e WatchSource:0}: Error finding container dd688ceca132d007a07f6d9d89531f8d9821aadd185047eb621e88bad3d0f01e: Status 404 returned error can't find the container with id dd688ceca132d007a07f6d9d89531f8d9821aadd185047eb621e88bad3d0f01e Jul 12 18:26:17.170262 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:17.170225 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8q6bw"] Jul 12 18:26:17.173233 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:26:17.173205 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7abc23a1_1bde_4046_a60d_cf3e93e64f63.slice/crio-fcf7dc32461e0f27b00cfdf582f831e1b5a3673ca653666c749f88b0ffaf893a WatchSource:0}: Error finding container fcf7dc32461e0f27b00cfdf582f831e1b5a3673ca653666c749f88b0ffaf893a: Status 404 returned error can't find the container with id fcf7dc32461e0f27b00cfdf582f831e1b5a3673ca653666c749f88b0ffaf893a Jul 12 18:26:17.173531 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:17.173505 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lphwv"] Jul 12 18:26:17.176594 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:26:17.176569 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb62dd7e_0d0f_4128_87dd_3158aa45a396.slice/crio-9734a8eda2f350dde4a67b4ee169b4d782f4b3d0619062e14be61007ac24b06e WatchSource:0}: Error finding container 9734a8eda2f350dde4a67b4ee169b4d782f4b3d0619062e14be61007ac24b06e: Status 404 returned error can't find the container with id 9734a8eda2f350dde4a67b4ee169b4d782f4b3d0619062e14be61007ac24b06e Jul 12 18:26:17.664201 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:17.664156 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r5d4d" event={"ID":"e98e68e3-3c5c-45b0-9817-a05d41b96502","Type":"ContainerStarted","Data":"32cccb1ce18fb881ccae41fa212d66669928cbd10eee62e7c10d7423e44fecb2"} Jul 12 18:26:17.665906 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:17.665876 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8q6bw" event={"ID":"7abc23a1-1bde-4046-a60d-cf3e93e64f63","Type":"ContainerStarted","Data":"6d3afd48bcf17be81d32a1a4648eb9d3926bc6c2be150e2464e2fd05882d4672"} Jul 12 18:26:17.666043 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:17.665922 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8q6bw" event={"ID":"7abc23a1-1bde-4046-a60d-cf3e93e64f63","Type":"ContainerStarted","Data":"fcf7dc32461e0f27b00cfdf582f831e1b5a3673ca653666c749f88b0ffaf893a"} Jul 12 18:26:17.666096 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:17.666070 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:26:17.667192 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:17.667169 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j5bpc" event={"ID":"6ad0a79f-69ee-49e8-90eb-c27482619b69","Type":"ContainerStarted","Data":"8e915ca20c2bfd56d62052a806509a23d6262c98b1cf744e6ebfe8002f901a4d"} Jul 12 18:26:17.668525 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:17.668501 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rgkpj" event={"ID":"87644bbc-d561-4ca5-8b2b-38bf70f01efb","Type":"ContainerStarted","Data":"5db5d057f1d45473d74cdca18c62fa25bfbb8c45e193710e9cfdc6585c4d3b60"} Jul 12 18:26:17.669850 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:17.669803 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lphwv" event={"ID":"cb62dd7e-0d0f-4128-87dd-3158aa45a396","Type":"ContainerStarted","Data":"9734a8eda2f350dde4a67b4ee169b4d782f4b3d0619062e14be61007ac24b06e"} Jul 12 18:26:17.671519 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:17.671482 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" event={"ID":"cfdb85db-878a-43b4-af66-ebe34aea186a","Type":"ContainerStarted","Data":"49d9255c6e1d4c3a82d21fb103aa301fdaeae3e9fe9aa76940223606aafe6686"} Jul 12 18:26:17.671627 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:17.671524 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" event={"ID":"cfdb85db-878a-43b4-af66-ebe34aea186a","Type":"ContainerStarted","Data":"dd688ceca132d007a07f6d9d89531f8d9821aadd185047eb621e88bad3d0f01e"} Jul 12 18:26:17.671627 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:17.671552 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:26:17.684332 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:17.684026 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8q6bw" podStartSLOduration=67.684007444 podStartE2EDuration="1m7.684007444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:17.682720079 +0000 UTC m=+68.882187111" watchObservedRunningTime="2026-07-12 18:26:17.684007444 +0000 UTC m=+68.883474455" Jul 12 18:26:17.718303 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:17.718222 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" podStartSLOduration=67.718205106 podStartE2EDuration="1m7.718205106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:17.716993062 +0000 UTC m=+68.916460092" watchObservedRunningTime="2026-07-12 18:26:17.718205106 +0000 UTC m=+68.917672136" Jul 12 18:26:18.678383 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:18.678341 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r5d4d" event={"ID":"e98e68e3-3c5c-45b0-9817-a05d41b96502","Type":"ContainerStarted","Data":"d44f0c4c9bc6b30aedb90d3fdb1011a4dc18d92fcb277e27d9eff82de6e62ffd"} Jul 12 18:26:18.926766 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:18.926312 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg"] Jul 12 18:26:18.951139 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:18.949270 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg"] Jul 12 18:26:18.951139 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:18.949389 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" Jul 12 18:26:18.952353 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:18.952327 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 12 18:26:18.952871 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:18.952835 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 12 18:26:18.953764 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:18.953327 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 12 18:26:18.953764 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:18.953593 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 12 18:26:18.996666 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:18.996618 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-gzh72"] Jul 12 18:26:19.024176 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.023974 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-gzh72"] Jul 12 18:26:19.024176 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.024037 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-gzh72" Jul 12 18:26:19.026329 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.026310 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 12 18:26:19.026589 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.026565 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 12 18:26:19.026697 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.026654 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-n2kg4\"" Jul 12 18:26:19.043011 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.042968 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/eb22bb2b-34f7-48b7-bed7-102e87b2e3fa-klusterlet-config\") pod \"klusterlet-addon-workmgr-6755797779-x57fg\" (UID: \"eb22bb2b-34f7-48b7-bed7-102e87b2e3fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" Jul 12 18:26:19.043189 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.043087 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb22bb2b-34f7-48b7-bed7-102e87b2e3fa-tmp\") pod \"klusterlet-addon-workmgr-6755797779-x57fg\" (UID: \"eb22bb2b-34f7-48b7-bed7-102e87b2e3fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" Jul 12 18:26:19.043189 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.043115 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6mdk\" (UniqueName: \"kubernetes.io/projected/eb22bb2b-34f7-48b7-bed7-102e87b2e3fa-kube-api-access-q6mdk\") pod \"klusterlet-addon-workmgr-6755797779-x57fg\" (UID: \"eb22bb2b-34f7-48b7-bed7-102e87b2e3fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" Jul 12 18:26:19.143840 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.143750 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/eb22bb2b-34f7-48b7-bed7-102e87b2e3fa-klusterlet-config\") pod \"klusterlet-addon-workmgr-6755797779-x57fg\" (UID: \"eb22bb2b-34f7-48b7-bed7-102e87b2e3fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" Jul 12 18:26:19.144031 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.143848 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb22bb2b-34f7-48b7-bed7-102e87b2e3fa-tmp\") pod \"klusterlet-addon-workmgr-6755797779-x57fg\" (UID: \"eb22bb2b-34f7-48b7-bed7-102e87b2e3fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" Jul 12 18:26:19.144031 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.143879 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6mdk\" (UniqueName: \"kubernetes.io/projected/eb22bb2b-34f7-48b7-bed7-102e87b2e3fa-kube-api-access-q6mdk\") pod \"klusterlet-addon-workmgr-6755797779-x57fg\" (UID: \"eb22bb2b-34f7-48b7-bed7-102e87b2e3fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" Jul 12 18:26:19.144031 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.143910 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxbsr\" (UniqueName: \"kubernetes.io/projected/484975e5-3874-48ae-a247-8c9b36bc813d-kube-api-access-lxbsr\") pod \"downloads-68c9cbb497-gzh72\" (UID: \"484975e5-3874-48ae-a247-8c9b36bc813d\") " pod="openshift-console/downloads-68c9cbb497-gzh72" Jul 12 18:26:19.144338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.144319 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb22bb2b-34f7-48b7-bed7-102e87b2e3fa-tmp\") pod \"klusterlet-addon-workmgr-6755797779-x57fg\" (UID: \"eb22bb2b-34f7-48b7-bed7-102e87b2e3fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" Jul 12 18:26:19.146550 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.146527 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/eb22bb2b-34f7-48b7-bed7-102e87b2e3fa-klusterlet-config\") pod \"klusterlet-addon-workmgr-6755797779-x57fg\" (UID: \"eb22bb2b-34f7-48b7-bed7-102e87b2e3fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" Jul 12 18:26:19.153097 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.153046 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6mdk\" (UniqueName: \"kubernetes.io/projected/eb22bb2b-34f7-48b7-bed7-102e87b2e3fa-kube-api-access-q6mdk\") pod \"klusterlet-addon-workmgr-6755797779-x57fg\" (UID: \"eb22bb2b-34f7-48b7-bed7-102e87b2e3fa\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" Jul 12 18:26:19.245120 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.245085 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxbsr\" (UniqueName: \"kubernetes.io/projected/484975e5-3874-48ae-a247-8c9b36bc813d-kube-api-access-lxbsr\") pod \"downloads-68c9cbb497-gzh72\" (UID: \"484975e5-3874-48ae-a247-8c9b36bc813d\") " pod="openshift-console/downloads-68c9cbb497-gzh72" Jul 12 18:26:19.254125 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.254062 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxbsr\" (UniqueName: \"kubernetes.io/projected/484975e5-3874-48ae-a247-8c9b36bc813d-kube-api-access-lxbsr\") pod \"downloads-68c9cbb497-gzh72\" (UID: \"484975e5-3874-48ae-a247-8c9b36bc813d\") " pod="openshift-console/downloads-68c9cbb497-gzh72" Jul 12 18:26:19.276153 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.276121 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" Jul 12 18:26:19.334357 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.334310 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-gzh72" Jul 12 18:26:19.685577 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.685538 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rgkpj" event={"ID":"87644bbc-d561-4ca5-8b2b-38bf70f01efb","Type":"ContainerStarted","Data":"14751d920e43cf94ec47c3da6c417d852908a62dae0dc59491f9ca5c53c51a8f"} Jul 12 18:26:19.976368 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:19.976331 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-gzh72"] Jul 12 18:26:19.982561 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:26:19.982525 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod484975e5_3874_48ae_a247_8c9b36bc813d.slice/crio-83018b2e8853540b0be62afeda7a43c13322607a0bbb79c7794e073ff246ab35 WatchSource:0}: Error finding container 83018b2e8853540b0be62afeda7a43c13322607a0bbb79c7794e073ff246ab35: Status 404 returned error can't find the container with id 83018b2e8853540b0be62afeda7a43c13322607a0bbb79c7794e073ff246ab35 Jul 12 18:26:20.000602 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.000562 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg"] Jul 12 18:26:20.006826 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:26:20.006790 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb22bb2b_34f7_48b7_bed7_102e87b2e3fa.slice/crio-09ac1b5a49667256012dabde9a7ef4ade83d51e0359aaf005de77c9eb1c78ba7 WatchSource:0}: Error finding container 09ac1b5a49667256012dabde9a7ef4ade83d51e0359aaf005de77c9eb1c78ba7: Status 404 returned error can't find the container with id 09ac1b5a49667256012dabde9a7ef4ade83d51e0359aaf005de77c9eb1c78ba7 Jul 12 18:26:20.357054 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.357007 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-kzl7d\" (UID: \"d1e3d19c-3412-49a9-a51f-4658c02209bd\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" Jul 12 18:26:20.357250 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.357066 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-qrltf\" (UID: \"b5d716f4-9338-4351-a5fc-46b1513626d0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" Jul 12 18:26:20.360127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.360097 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b5d716f4-9338-4351-a5fc-46b1513626d0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-qrltf\" (UID: \"b5d716f4-9338-4351-a5fc-46b1513626d0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" Jul 12 18:26:20.360127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.360110 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1e3d19c-3412-49a9-a51f-4658c02209bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-kzl7d\" (UID: \"d1e3d19c-3412-49a9-a51f-4658c02209bd\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" Jul 12 18:26:20.458408 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.458374 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:26:20.458589 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.458460 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:26:20.459137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.459113 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80edfd93-3c7a-4356-9c21-6b1376df14fb-service-ca-bundle\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:26:20.461033 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.461013 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80edfd93-3c7a-4356-9c21-6b1376df14fb-metrics-certs\") pod \"router-default-549c4cff84-xqpgj\" (UID: \"80edfd93-3c7a-4356-9c21-6b1376df14fb\") " pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:26:20.488118 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.488084 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-k79gm\"" Jul 12 18:26:20.496004 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.495967 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" Jul 12 18:26:20.559270 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.559236 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:26:20.562246 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.562201 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a33fa508-623e-4693-88b0-51eb8cc79715-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-642k2\" (UID: \"a33fa508-623e-4693-88b0-51eb8cc79715\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:26:20.581199 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.581159 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qvvn7\"" Jul 12 18:26:20.590169 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.590135 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" Jul 12 18:26:20.603490 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.603461 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-z26dw\"" Jul 12 18:26:20.611741 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.611662 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:26:20.691518 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.691479 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rgkpj" event={"ID":"87644bbc-d561-4ca5-8b2b-38bf70f01efb","Type":"ContainerStarted","Data":"ada14156163c5da84dc5dc25b88bf0aba1139361e4a15366d070051bcff515b5"} Jul 12 18:26:20.692838 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.692805 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" event={"ID":"eb22bb2b-34f7-48b7-bed7-102e87b2e3fa","Type":"ContainerStarted","Data":"09ac1b5a49667256012dabde9a7ef4ade83d51e0359aaf005de77c9eb1c78ba7"} Jul 12 18:26:20.694239 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.694213 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-gzh72" event={"ID":"484975e5-3874-48ae-a247-8c9b36bc813d","Type":"ContainerStarted","Data":"83018b2e8853540b0be62afeda7a43c13322607a0bbb79c7794e073ff246ab35"} Jul 12 18:26:20.696347 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.696306 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lphwv" event={"ID":"cb62dd7e-0d0f-4128-87dd-3158aa45a396","Type":"ContainerStarted","Data":"d285ae527309988c3ec7b2208b2aeae7d66c0ed936c7c464e36466d45b7b61e2"} Jul 12 18:26:20.696461 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.696354 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lphwv" event={"ID":"cb62dd7e-0d0f-4128-87dd-3158aa45a396","Type":"ContainerStarted","Data":"9dc16751c9c278cd54d9ebe65608361d2f4e44e3ac2d0a93bb4da153eabb2e8a"} Jul 12 18:26:20.696461 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.696444 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lphwv" Jul 12 18:26:20.707353 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.707286 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rgkpj" podStartSLOduration=68.859636751 podStartE2EDuration="1m10.707265142s" totalImagesPullingTime="1.847628391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:20.706099557 +0000 UTC m=+71.905566589" watchObservedRunningTime="2026-07-12 18:26:20.707265142 +0000 UTC m=+71.906732175" Jul 12 18:26:20.722644 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.722578 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lphwv" podStartSLOduration=34.07761038 podStartE2EDuration="36.722558921s" totalImagesPullingTime="2.644948541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:20.721676038 +0000 UTC m=+71.921143069" watchObservedRunningTime="2026-07-12 18:26:20.722558921 +0000 UTC m=+71.922025953" Jul 12 18:26:20.723640 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.723611 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lz5wn\"" Jul 12 18:26:20.731378 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:20.731352 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" Jul 12 18:26:23.150012 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:23.149980 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-549c4cff84-xqpgj"] Jul 12 18:26:23.212268 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:23.212224 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf"] Jul 12 18:26:23.384737 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:23.384648 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-kzl7d"] Jul 12 18:26:23.387000 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:23.386907 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2"] Jul 12 18:26:23.715103 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:23.715032 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" event={"ID":"d1e3d19c-3412-49a9-a51f-4658c02209bd","Type":"ContainerStarted","Data":"edeed41d8a1d40eeb3da1ec6e2cad227e18c82da8c6dcef4f5ba2c40dc8b1c9d"} Jul 12 18:26:23.717694 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:23.717656 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" event={"ID":"a33fa508-623e-4693-88b0-51eb8cc79715","Type":"ContainerStarted","Data":"86aa1d7a6235511426aa89780176c6ba05e1f0fe9d6432fd29dc9ec3327286b7"} Jul 12 18:26:23.720574 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:23.720540 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r5d4d" event={"ID":"e98e68e3-3c5c-45b0-9817-a05d41b96502","Type":"ContainerStarted","Data":"009f1ef55d9e42c9cbdcfc035447a85bf425d69406e0e385785539361a23f2d3"} Jul 12 18:26:23.723259 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:23.723190 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" event={"ID":"b5d716f4-9338-4351-a5fc-46b1513626d0","Type":"ContainerStarted","Data":"fdd94b0463c459ad7851b33da26ca1e9a2ac51bccbc554eec25dc8e403bb66e5"} Jul 12 18:26:23.724955 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:23.724903 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j5bpc" event={"ID":"6ad0a79f-69ee-49e8-90eb-c27482619b69","Type":"ContainerStarted","Data":"0f3cd49345b1f32374a806bf665ff2bb058f5c013481e08f90209c48632c738a"} Jul 12 18:26:23.728173 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:23.727639 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-549c4cff84-xqpgj" event={"ID":"80edfd93-3c7a-4356-9c21-6b1376df14fb","Type":"ContainerStarted","Data":"7bc7c9220ae0fc6d449f1f737cc1415a1e7e758847fd438445a929c6435baeba"} Jul 12 18:26:23.728173 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:23.727680 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-549c4cff84-xqpgj" event={"ID":"80edfd93-3c7a-4356-9c21-6b1376df14fb","Type":"ContainerStarted","Data":"8ad4b281e84562c7b8c3740eff2187270f371067bdc7238b503e87b29783929e"} Jul 12 18:26:23.765143 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:23.765083 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-549c4cff84-xqpgj" podStartSLOduration=35.765059384 podStartE2EDuration="35.765059384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:23.764416942 +0000 UTC m=+74.963884004" watchObservedRunningTime="2026-07-12 18:26:23.765059384 +0000 UTC m=+74.964526415" Jul 12 18:26:23.766576 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:23.766524 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r5d4d" podStartSLOduration=17.955813909 podStartE2EDuration="27.766508411s" totalImagesPullingTime="9.810694502s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:23.744130829 +0000 UTC m=+74.943597860" watchObservedRunningTime="2026-07-12 18:26:23.766508411 +0000 UTC m=+74.965975443" Jul 12 18:26:23.782763 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:23.781329 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j5bpc" podStartSLOduration=67.861752951 podStartE2EDuration="1m13.781305069s" totalImagesPullingTime="5.919552118s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:23.780231957 +0000 UTC m=+74.979699008" watchObservedRunningTime="2026-07-12 18:26:23.781305069 +0000 UTC m=+74.980772102" Jul 12 18:26:24.612775 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:24.612724 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:26:24.616551 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:24.616521 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:26:24.734013 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:24.733978 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:26:24.735803 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:24.735602 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-549c4cff84-xqpgj" Jul 12 18:26:29.764866 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:29.764828 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" event={"ID":"eb22bb2b-34f7-48b7-bed7-102e87b2e3fa","Type":"ContainerStarted","Data":"afc701ab84cc7ad1f06b29484e7cfc5ca3025137512743614ac253931c9aa277"} Jul 12 18:26:29.765472 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:29.765388 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" Jul 12 18:26:29.767754 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:29.767724 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" event={"ID":"d1e3d19c-3412-49a9-a51f-4658c02209bd","Type":"ContainerStarted","Data":"c2dda8f434b2db9d9bd15efae677243332bb61fd4519a8b1580cb270f48ec6c7"} Jul 12 18:26:29.768258 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:29.768237 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" Jul 12 18:26:29.770042 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:29.770018 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" event={"ID":"a33fa508-623e-4693-88b0-51eb8cc79715","Type":"ContainerStarted","Data":"389b29fb76e278f7349013a5517ee56dd95b15711fe8da9f7b07e5149c7981de"} Jul 12 18:26:29.772165 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:29.772141 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" event={"ID":"b5d716f4-9338-4351-a5fc-46b1513626d0","Type":"ContainerStarted","Data":"914fcd489fbce28f4d37588febb8db125f1e26888d4a18e7112a4a8b25057970"} Jul 12 18:26:29.772320 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:29.772300 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" event={"ID":"b5d716f4-9338-4351-a5fc-46b1513626d0","Type":"ContainerStarted","Data":"a5f791e7e93ef138b067b28d310470592a7e4d585cbc29d6f3bddc5ac8051d48"} Jul 12 18:26:29.783318 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:29.783257 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6755797779-x57fg" podStartSLOduration=2.52142487 podStartE2EDuration="11.783235369s" totalImagesPullingTime="9.261810499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:29.781791991 +0000 UTC m=+80.981259019" watchObservedRunningTime="2026-07-12 18:26:29.783235369 +0000 UTC m=+80.982702400" Jul 12 18:26:29.819330 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:29.819171 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-qrltf" podStartSLOduration=35.852298054 podStartE2EDuration="41.819098277s" totalImagesPullingTime="5.966800223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:29.818260875 +0000 UTC m=+81.017727906" watchObservedRunningTime="2026-07-12 18:26:29.819098277 +0000 UTC m=+81.018565309" Jul 12 18:26:29.845911 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:29.845176 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-642k2" podStartSLOduration=35.993979258 podStartE2EDuration="41.84515222s" totalImagesPullingTime="5.851172962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:29.844108041 +0000 UTC m=+81.043575083" watchObservedRunningTime="2026-07-12 18:26:29.84515222 +0000 UTC m=+81.044619251" Jul 12 18:26:29.867414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:29.867361 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-kzl7d" podStartSLOduration=36.015049698 podStartE2EDuration="41.867346218s" totalImagesPullingTime="5.85229652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:29.866115584 +0000 UTC m=+81.065582616" watchObservedRunningTime="2026-07-12 18:26:29.867346218 +0000 UTC m=+81.066813248" Jul 12 18:26:30.702707 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:30.702674 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lphwv" Jul 12 18:26:38.684691 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:38.684557 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:26:40.300058 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.300015 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b6f7f"] Jul 12 18:26:40.322663 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.322631 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.326157 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.326134 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 12 18:26:40.326302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.326131 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 12 18:26:40.326717 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.326693 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 12 18:26:40.326829 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.326728 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wnhfx\"" Jul 12 18:26:40.326829 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.326731 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 12 18:26:40.452955 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.452898 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-tls\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.453127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.452985 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d3f6345-96f3-40ba-9a77-73c5e424e968-metrics-client-ca\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.453127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.453012 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-textfile\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.453127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.453045 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bwhd\" (UniqueName: \"kubernetes.io/projected/8d3f6345-96f3-40ba-9a77-73c5e424e968-kube-api-access-8bwhd\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.453127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.453069 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-wtmp\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.453127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.453086 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.453324 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.453132 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8d3f6345-96f3-40ba-9a77-73c5e424e968-root\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.453324 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.453158 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d3f6345-96f3-40ba-9a77-73c5e424e968-sys\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.453324 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.453183 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-accelerators-collector-config\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.553965 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.553850 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bwhd\" (UniqueName: \"kubernetes.io/projected/8d3f6345-96f3-40ba-9a77-73c5e424e968-kube-api-access-8bwhd\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.553965 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.553909 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-wtmp\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.553965 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.553963 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.554234 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.553994 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8d3f6345-96f3-40ba-9a77-73c5e424e968-root\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.554234 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.554023 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d3f6345-96f3-40ba-9a77-73c5e424e968-sys\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.554234 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.554064 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-accelerators-collector-config\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.554234 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.554119 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-wtmp\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.554234 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.554128 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-tls\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.554234 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:40.554190 2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 12 18:26:40.554234 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.554207 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d3f6345-96f3-40ba-9a77-73c5e424e968-metrics-client-ca\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.554563 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:26:40.554250 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-tls podName:8d3f6345-96f3-40ba-9a77-73c5e424e968 nodeName:}" failed. No retries permitted until 2026-07-12 18:26:41.054230602 +0000 UTC m=+92.253697610 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-tls") pod "node-exporter-b6f7f" (UID: "8d3f6345-96f3-40ba-9a77-73c5e424e968") : secret "node-exporter-tls" not found Jul 12 18:26:40.554563 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.554268 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-textfile\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.554563 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.554325 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d3f6345-96f3-40ba-9a77-73c5e424e968-sys\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.554563 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.554346 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8d3f6345-96f3-40ba-9a77-73c5e424e968-root\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.554889 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.554864 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d3f6345-96f3-40ba-9a77-73c5e424e968-metrics-client-ca\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.554999 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.554889 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-accelerators-collector-config\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.555072 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.555051 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-textfile\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.556990 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.556963 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.563346 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.563307 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bwhd\" (UniqueName: \"kubernetes.io/projected/8d3f6345-96f3-40ba-9a77-73c5e424e968-kube-api-access-8bwhd\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:40.822222 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.822118 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-gzh72" event={"ID":"484975e5-3874-48ae-a247-8c9b36bc813d","Type":"ContainerStarted","Data":"2365c3e5245c8a8813e7a3a2076c783e96b967e3c0ebdb2d0f69b62bce83aa98"} Jul 12 18:26:40.822390 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.822351 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-gzh72" Jul 12 18:26:40.833398 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.833361 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-gzh72" Jul 12 18:26:40.840363 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:40.840292 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-gzh72" podStartSLOduration=2.488597191 podStartE2EDuration="22.840270487s" totalImagesPullingTime="20.351673296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:40.838638885 +0000 UTC m=+92.038105916" watchObservedRunningTime="2026-07-12 18:26:40.840270487 +0000 UTC m=+92.039737518" Jul 12 18:26:41.057989 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:41.057919 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-tls\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:41.060835 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:41.060797 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d3f6345-96f3-40ba-9a77-73c5e424e968-node-exporter-tls\") pod \"node-exporter-b6f7f\" (UID: \"8d3f6345-96f3-40ba-9a77-73c5e424e968\") " pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:41.233462 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:41.232792 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b6f7f" Jul 12 18:26:41.275532 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:26:41.275498 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d3f6345_96f3_40ba_9a77_73c5e424e968.slice/crio-3f35eeb528e458c022c7e4527db0c9ef349d75e1872cfa805ba28426978257af WatchSource:0}: Error finding container 3f35eeb528e458c022c7e4527db0c9ef349d75e1872cfa805ba28426978257af: Status 404 returned error can't find the container with id 3f35eeb528e458c022c7e4527db0c9ef349d75e1872cfa805ba28426978257af Jul 12 18:26:41.835335 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:41.835247 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b6f7f" event={"ID":"8d3f6345-96f3-40ba-9a77-73c5e424e968","Type":"ContainerStarted","Data":"3f35eeb528e458c022c7e4527db0c9ef349d75e1872cfa805ba28426978257af"} Jul 12 18:26:41.955956 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:41.955463 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67bd449f9f-lf78m"] Jul 12 18:26:43.846194 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:43.846157 2585 generic.go:356] "Generic (PLEG): container finished" podID="8d3f6345-96f3-40ba-9a77-73c5e424e968" containerID="73397110a55e11513e72a825fe9bab1f71a68973a35532a6cec1fe43011cfafe" exitCode=0 Jul 12 18:26:43.846684 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:43.846212 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b6f7f" event={"ID":"8d3f6345-96f3-40ba-9a77-73c5e424e968","Type":"ContainerDied","Data":"73397110a55e11513e72a825fe9bab1f71a68973a35532a6cec1fe43011cfafe"} Jul 12 18:26:44.852937 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:44.852899 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b6f7f" event={"ID":"8d3f6345-96f3-40ba-9a77-73c5e424e968","Type":"ContainerStarted","Data":"ecf1c38db5fccd32966ad9a36a3ebb90ea578a64f13a23c6b797149f03531960"} Jul 12 18:26:44.853462 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:44.852978 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b6f7f" event={"ID":"8d3f6345-96f3-40ba-9a77-73c5e424e968","Type":"ContainerStarted","Data":"e926da7190a7d80240d4500eb223b964b120daf13ef33a74d1634509a8301fa6"} Jul 12 18:26:44.873517 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:44.873452 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b6f7f" podStartSLOduration=3.111243636 podStartE2EDuration="4.873435522s" totalImagesPullingTime="1.661489545s" totalInitContainerRuntime="100.702341ms" isStatefulPod=true podCreationTimestamp="2026-07-12 18:26:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:44.871475188 +0000 UTC m=+96.070942230" watchObservedRunningTime="2026-07-12 18:26:44.873435522 +0000 UTC m=+96.072902551" Jul 12 18:26:48.681534 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:26:48.681495 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8q6bw" Jul 12 18:27:06.986731 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:06.986687 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" podUID="cfdb85db-878a-43b4-af66-ebe34aea186a" containerName="registry" containerID="cri-o://49d9255c6e1d4c3a82d21fb103aa301fdaeae3e9fe9aa76940223606aafe6686" gracePeriod=30 Jul 12 18:27:07.258801 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.258719 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:27:07.388374 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.388328 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls\") pod \"cfdb85db-878a-43b4-af66-ebe34aea186a\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " Jul 12 18:27:07.388374 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.388380 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cfdb85db-878a-43b4-af66-ebe34aea186a-image-registry-private-configuration\") pod \"cfdb85db-878a-43b4-af66-ebe34aea186a\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " Jul 12 18:27:07.388621 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.388438 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-bound-sa-token\") pod \"cfdb85db-878a-43b4-af66-ebe34aea186a\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " Jul 12 18:27:07.388621 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.388464 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-certificates\") pod \"cfdb85db-878a-43b4-af66-ebe34aea186a\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " Jul 12 18:27:07.388621 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.388495 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdkrb\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-kube-api-access-bdkrb\") pod \"cfdb85db-878a-43b4-af66-ebe34aea186a\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " Jul 12 18:27:07.388621 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.388524 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cfdb85db-878a-43b4-af66-ebe34aea186a-ca-trust-extracted\") pod \"cfdb85db-878a-43b4-af66-ebe34aea186a\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " Jul 12 18:27:07.388621 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.388549 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cfdb85db-878a-43b4-af66-ebe34aea186a-installation-pull-secrets\") pod \"cfdb85db-878a-43b4-af66-ebe34aea186a\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " Jul 12 18:27:07.388621 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.388575 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cfdb85db-878a-43b4-af66-ebe34aea186a-trusted-ca\") pod \"cfdb85db-878a-43b4-af66-ebe34aea186a\" (UID: \"cfdb85db-878a-43b4-af66-ebe34aea186a\") " Jul 12 18:27:07.389103 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.388964 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cfdb85db-878a-43b4-af66-ebe34aea186a" (UID: "cfdb85db-878a-43b4-af66-ebe34aea186a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:27:07.389289 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.389196 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfdb85db-878a-43b4-af66-ebe34aea186a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cfdb85db-878a-43b4-af66-ebe34aea186a" (UID: "cfdb85db-878a-43b4-af66-ebe34aea186a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:27:07.391426 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.391357 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfdb85db-878a-43b4-af66-ebe34aea186a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cfdb85db-878a-43b4-af66-ebe34aea186a" (UID: "cfdb85db-878a-43b4-af66-ebe34aea186a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:27:07.391596 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.391451 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cfdb85db-878a-43b4-af66-ebe34aea186a" (UID: "cfdb85db-878a-43b4-af66-ebe34aea186a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:27:07.391596 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.391518 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfdb85db-878a-43b4-af66-ebe34aea186a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cfdb85db-878a-43b4-af66-ebe34aea186a" (UID: "cfdb85db-878a-43b4-af66-ebe34aea186a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:27:07.391596 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.391561 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cfdb85db-878a-43b4-af66-ebe34aea186a" (UID: "cfdb85db-878a-43b4-af66-ebe34aea186a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:27:07.391757 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.391679 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-kube-api-access-bdkrb" (OuterVolumeSpecName: "kube-api-access-bdkrb") pod "cfdb85db-878a-43b4-af66-ebe34aea186a" (UID: "cfdb85db-878a-43b4-af66-ebe34aea186a"). InnerVolumeSpecName "kube-api-access-bdkrb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:27:07.397577 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.397547 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfdb85db-878a-43b4-af66-ebe34aea186a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cfdb85db-878a-43b4-af66-ebe34aea186a" (UID: "cfdb85db-878a-43b4-af66-ebe34aea186a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:27:07.489867 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.489820 2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-tls\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:27:07.489867 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.489861 2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cfdb85db-878a-43b4-af66-ebe34aea186a-image-registry-private-configuration\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:27:07.489867 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.489871 2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-bound-sa-token\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:27:07.490130 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.489880 2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cfdb85db-878a-43b4-af66-ebe34aea186a-registry-certificates\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:27:07.490130 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.489891 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdkrb\" (UniqueName: \"kubernetes.io/projected/cfdb85db-878a-43b4-af66-ebe34aea186a-kube-api-access-bdkrb\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:27:07.490130 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.489899 2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cfdb85db-878a-43b4-af66-ebe34aea186a-ca-trust-extracted\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:27:07.490130 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.489908 2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cfdb85db-878a-43b4-af66-ebe34aea186a-installation-pull-secrets\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:27:07.490130 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.489916 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cfdb85db-878a-43b4-af66-ebe34aea186a-trusted-ca\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:27:07.941108 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.941073 2585 generic.go:356] "Generic (PLEG): container finished" podID="cfdb85db-878a-43b4-af66-ebe34aea186a" containerID="49d9255c6e1d4c3a82d21fb103aa301fdaeae3e9fe9aa76940223606aafe6686" exitCode=0 Jul 12 18:27:07.941274 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.941137 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" Jul 12 18:27:07.941274 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.941154 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" event={"ID":"cfdb85db-878a-43b4-af66-ebe34aea186a","Type":"ContainerDied","Data":"49d9255c6e1d4c3a82d21fb103aa301fdaeae3e9fe9aa76940223606aafe6686"} Jul 12 18:27:07.941274 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.941207 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67bd449f9f-lf78m" event={"ID":"cfdb85db-878a-43b4-af66-ebe34aea186a","Type":"ContainerDied","Data":"dd688ceca132d007a07f6d9d89531f8d9821aadd185047eb621e88bad3d0f01e"} Jul 12 18:27:07.941274 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.941228 2585 scope.go:117] "RemoveContainer" containerID="49d9255c6e1d4c3a82d21fb103aa301fdaeae3e9fe9aa76940223606aafe6686" Jul 12 18:27:07.949982 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.949962 2585 scope.go:117] "RemoveContainer" containerID="49d9255c6e1d4c3a82d21fb103aa301fdaeae3e9fe9aa76940223606aafe6686" Jul 12 18:27:07.950255 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:27:07.950231 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49d9255c6e1d4c3a82d21fb103aa301fdaeae3e9fe9aa76940223606aafe6686\": container with ID starting with 49d9255c6e1d4c3a82d21fb103aa301fdaeae3e9fe9aa76940223606aafe6686 not found: ID does not exist" containerID="49d9255c6e1d4c3a82d21fb103aa301fdaeae3e9fe9aa76940223606aafe6686" Jul 12 18:27:07.950301 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.950263 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49d9255c6e1d4c3a82d21fb103aa301fdaeae3e9fe9aa76940223606aafe6686"} err="failed to get container status \"49d9255c6e1d4c3a82d21fb103aa301fdaeae3e9fe9aa76940223606aafe6686\": rpc error: code = NotFound desc = could not find container \"49d9255c6e1d4c3a82d21fb103aa301fdaeae3e9fe9aa76940223606aafe6686\": container with ID starting with 49d9255c6e1d4c3a82d21fb103aa301fdaeae3e9fe9aa76940223606aafe6686 not found: ID does not exist" Jul 12 18:27:07.961627 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.961601 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67bd449f9f-lf78m"] Jul 12 18:27:07.965719 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:07.965696 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67bd449f9f-lf78m"] Jul 12 18:27:09.303082 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:09.303051 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfdb85db-878a-43b4-af66-ebe34aea186a" path="/var/lib/kubelet/pods/cfdb85db-878a-43b4-af66-ebe34aea186a/volumes" Jul 12 18:27:11.961981 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:11.961949 2585 generic.go:356] "Generic (PLEG): container finished" podID="429df8ad-28e9-487c-a2d9-d66a44615bc4" containerID="9331c9ff47bcbec3f722a5f0c9fcc416de64f89f5f2baa2aecea4d85b21af390" exitCode=0 Jul 12 18:27:11.962379 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:11.962014 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" event={"ID":"429df8ad-28e9-487c-a2d9-d66a44615bc4","Type":"ContainerDied","Data":"9331c9ff47bcbec3f722a5f0c9fcc416de64f89f5f2baa2aecea4d85b21af390"} Jul 12 18:27:11.962379 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:11.962315 2585 scope.go:117] "RemoveContainer" containerID="9331c9ff47bcbec3f722a5f0c9fcc416de64f89f5f2baa2aecea4d85b21af390" Jul 12 18:27:12.968172 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:12.968130 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-pm7l6" event={"ID":"429df8ad-28e9-487c-a2d9-d66a44615bc4","Type":"ContainerStarted","Data":"54bf0f0d143a5cbd9f8636258fb029989922436e2bb9de76958b8842426e037c"} Jul 12 18:27:16.983492 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:16.983456 2585 generic.go:356] "Generic (PLEG): container finished" podID="b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2" containerID="d12d28703776285c9a422a8190c31852b11fa6df28a44280e9f1a371c74e1e83" exitCode=0 Jul 12 18:27:16.983872 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:16.983525 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" event={"ID":"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2","Type":"ContainerDied","Data":"d12d28703776285c9a422a8190c31852b11fa6df28a44280e9f1a371c74e1e83"} Jul 12 18:27:16.983921 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:16.983886 2585 scope.go:117] "RemoveContainer" containerID="d12d28703776285c9a422a8190c31852b11fa6df28a44280e9f1a371c74e1e83" Jul 12 18:27:17.990592 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:17.990550 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-96sxt" event={"ID":"b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2","Type":"ContainerStarted","Data":"ddf327357f9c3fc646c50d82de4cc1a5bf1cd1f6bfb2d19638f89225c0e7e93d"} Jul 12 18:27:18.140904 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:18.140870 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-549c4cff84-xqpgj_80edfd93-3c7a-4356-9c21-6b1376df14fb/router/0.log" Jul 12 18:27:18.158299 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:18.158266 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l98hb_9aef0f72-416e-4c37-98b8-a68e30a65ce6/serve-healthcheck-canary/0.log" Jul 12 18:27:26.021532 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:26.021489 2585 generic.go:356] "Generic (PLEG): container finished" podID="621b603b-b0fd-4925-bbc6-015e3fd4d097" containerID="f513f071e9ce88b3308c6ded41a7b0860c7241e7e60f272dd979c8d1962fc826" exitCode=0 Jul 12 18:27:26.021984 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:26.021559 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" event={"ID":"621b603b-b0fd-4925-bbc6-015e3fd4d097","Type":"ContainerDied","Data":"f513f071e9ce88b3308c6ded41a7b0860c7241e7e60f272dd979c8d1962fc826"} Jul 12 18:27:26.021984 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:26.021893 2585 scope.go:117] "RemoveContainer" containerID="f513f071e9ce88b3308c6ded41a7b0860c7241e7e60f272dd979c8d1962fc826" Jul 12 18:27:27.027246 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:27:27.027208 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-mqv6k" event={"ID":"621b603b-b0fd-4925-bbc6-015e3fd4d097","Type":"ContainerStarted","Data":"97278a9d93c867cece8a72bb51595d57afb816cf64c624fdda0a75c13108b28c"} Jul 12 18:28:39.071660 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.068660 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5"] Jul 12 18:28:39.071660 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.069394 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfdb85db-878a-43b4-af66-ebe34aea186a" containerName="registry" Jul 12 18:28:39.071660 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.069416 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfdb85db-878a-43b4-af66-ebe34aea186a" containerName="registry" Jul 12 18:28:39.071660 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.069597 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfdb85db-878a-43b4-af66-ebe34aea186a" containerName="registry" Jul 12 18:28:39.073329 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.073302 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" Jul 12 18:28:39.075969 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.075940 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 18:28:39.076995 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.076969 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jb68p\"" Jul 12 18:28:39.077154 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.077049 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 18:28:39.078807 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.078784 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5"] Jul 12 18:28:39.114984 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.114941 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5\" (UID: \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" Jul 12 18:28:39.115170 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.115000 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5\" (UID: \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" Jul 12 18:28:39.115170 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.115025 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqnp4\" (UniqueName: \"kubernetes.io/projected/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-kube-api-access-mqnp4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5\" (UID: \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" Jul 12 18:28:39.215884 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.215853 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5\" (UID: \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" Jul 12 18:28:39.216044 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.215905 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5\" (UID: \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" Jul 12 18:28:39.216044 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.215925 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqnp4\" (UniqueName: \"kubernetes.io/projected/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-kube-api-access-mqnp4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5\" (UID: \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" Jul 12 18:28:39.216295 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.216276 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5\" (UID: \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" Jul 12 18:28:39.216330 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.216301 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5\" (UID: \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" Jul 12 18:28:39.228411 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.228374 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqnp4\" (UniqueName: \"kubernetes.io/projected/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-kube-api-access-mqnp4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5\" (UID: \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" Jul 12 18:28:39.384138 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.384033 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" Jul 12 18:28:39.519672 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:39.519635 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5"] Jul 12 18:28:39.522942 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:28:39.522894 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f1aa8a2_8410_4f11_a83c_9f2d8c52b27e.slice/crio-3e650f4b8d6bbc5df0e56c06c50251d34eb82e43fa6d1d8c0947a81048ede7b0 WatchSource:0}: Error finding container 3e650f4b8d6bbc5df0e56c06c50251d34eb82e43fa6d1d8c0947a81048ede7b0: Status 404 returned error can't find the container with id 3e650f4b8d6bbc5df0e56c06c50251d34eb82e43fa6d1d8c0947a81048ede7b0 Jul 12 18:28:40.301722 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:40.301669 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" event={"ID":"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e","Type":"ContainerStarted","Data":"3e650f4b8d6bbc5df0e56c06c50251d34eb82e43fa6d1d8c0947a81048ede7b0"} Jul 12 18:28:47.332161 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:47.332124 2585 generic.go:356] "Generic (PLEG): container finished" podID="0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e" containerID="dd6b4faca435a1682be51c1226090298ce7f3e3281ad0a013a018b5453381fff" exitCode=0 Jul 12 18:28:47.332605 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:47.332219 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" event={"ID":"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e","Type":"ContainerDied","Data":"dd6b4faca435a1682be51c1226090298ce7f3e3281ad0a013a018b5453381fff"} Jul 12 18:28:49.342347 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:49.342314 2585 generic.go:356] "Generic (PLEG): container finished" podID="0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e" containerID="f98d5f1890c66a3223cf3ed458213cb576d7fb5d440e87ddf851686f1352edc6" exitCode=0 Jul 12 18:28:49.342738 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:49.342354 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" event={"ID":"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e","Type":"ContainerDied","Data":"f98d5f1890c66a3223cf3ed458213cb576d7fb5d440e87ddf851686f1352edc6"} Jul 12 18:28:57.378231 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:57.378196 2585 generic.go:356] "Generic (PLEG): container finished" podID="0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e" containerID="7ddb25276a9577670ecaa8f802c2b16095ecc32e2f5664f7864d5689dcf6cb57" exitCode=0 Jul 12 18:28:57.378633 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:57.378276 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" event={"ID":"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e","Type":"ContainerDied","Data":"7ddb25276a9577670ecaa8f802c2b16095ecc32e2f5664f7864d5689dcf6cb57"} Jul 12 18:28:58.514909 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:58.514876 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" Jul 12 18:28:58.548967 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:58.548908 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-util\") pod \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\" (UID: \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\") " Jul 12 18:28:58.549142 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:58.548985 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-bundle\") pod \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\" (UID: \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\") " Jul 12 18:28:58.549142 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:58.549041 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqnp4\" (UniqueName: \"kubernetes.io/projected/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-kube-api-access-mqnp4\") pod \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\" (UID: \"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e\") " Jul 12 18:28:58.549808 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:58.549771 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-bundle" (OuterVolumeSpecName: "bundle") pod "0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e" (UID: "0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:28:58.551307 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:58.551278 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-kube-api-access-mqnp4" (OuterVolumeSpecName: "kube-api-access-mqnp4") pod "0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e" (UID: "0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e"). InnerVolumeSpecName "kube-api-access-mqnp4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:28:58.555211 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:58.555177 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-util" (OuterVolumeSpecName: "util") pod "0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e" (UID: "0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:28:58.649798 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:58.649687 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqnp4\" (UniqueName: \"kubernetes.io/projected/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-kube-api-access-mqnp4\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:28:58.649798 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:58.649742 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-util\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:28:58.649798 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:58.649754 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e-bundle\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:28:59.387344 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:59.387307 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" event={"ID":"0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e","Type":"ContainerDied","Data":"3e650f4b8d6bbc5df0e56c06c50251d34eb82e43fa6d1d8c0947a81048ede7b0"} Jul 12 18:28:59.387344 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:59.387343 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f9rj5" Jul 12 18:28:59.387553 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:28:59.387359 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e650f4b8d6bbc5df0e56c06c50251d34eb82e43fa6d1d8c0947a81048ede7b0" Jul 12 18:29:07.068011 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.067974 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64j9v"] Jul 12 18:29:07.068426 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.068253 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e" containerName="util" Jul 12 18:29:07.068426 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.068263 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e" containerName="util" Jul 12 18:29:07.068426 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.068271 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e" containerName="extract" Jul 12 18:29:07.068426 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.068277 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e" containerName="extract" Jul 12 18:29:07.068426 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.068287 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e" containerName="pull" Jul 12 18:29:07.068426 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.068293 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e" containerName="pull" Jul 12 18:29:07.068426 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.068370 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f1aa8a2-8410-4f11-a83c-9f2d8c52b27e" containerName="extract" Jul 12 18:29:07.071187 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.071167 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64j9v" Jul 12 18:29:07.073700 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.073675 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 12 18:29:07.073700 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.073692 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-rc7nt\"" Jul 12 18:29:07.073901 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.073734 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 12 18:29:07.082022 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.081993 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64j9v"] Jul 12 18:29:07.111172 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.111133 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj872\" (UniqueName: \"kubernetes.io/projected/6fb3be6c-d0ce-4185-8927-bc788b079877-kube-api-access-xj872\") pod \"cert-manager-operator-controller-manager-55c444bf86-64j9v\" (UID: \"6fb3be6c-d0ce-4185-8927-bc788b079877\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64j9v" Jul 12 18:29:07.111361 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.111193 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fb3be6c-d0ce-4185-8927-bc788b079877-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-64j9v\" (UID: \"6fb3be6c-d0ce-4185-8927-bc788b079877\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64j9v" Jul 12 18:29:07.211979 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.211916 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fb3be6c-d0ce-4185-8927-bc788b079877-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-64j9v\" (UID: \"6fb3be6c-d0ce-4185-8927-bc788b079877\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64j9v" Jul 12 18:29:07.212141 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.212025 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj872\" (UniqueName: \"kubernetes.io/projected/6fb3be6c-d0ce-4185-8927-bc788b079877-kube-api-access-xj872\") pod \"cert-manager-operator-controller-manager-55c444bf86-64j9v\" (UID: \"6fb3be6c-d0ce-4185-8927-bc788b079877\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64j9v" Jul 12 18:29:07.212365 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.212348 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fb3be6c-d0ce-4185-8927-bc788b079877-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-64j9v\" (UID: \"6fb3be6c-d0ce-4185-8927-bc788b079877\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64j9v" Jul 12 18:29:07.220979 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.220925 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj872\" (UniqueName: \"kubernetes.io/projected/6fb3be6c-d0ce-4185-8927-bc788b079877-kube-api-access-xj872\") pod \"cert-manager-operator-controller-manager-55c444bf86-64j9v\" (UID: \"6fb3be6c-d0ce-4185-8927-bc788b079877\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64j9v" Jul 12 18:29:07.380866 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.380778 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64j9v" Jul 12 18:29:07.522676 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:07.522638 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64j9v"] Jul 12 18:29:07.528128 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:29:07.528092 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb3be6c_d0ce_4185_8927_bc788b079877.slice/crio-697ff62ffea0afddffbe818dc94c7f363f95a3e19d630d4705fc1276dbcf2375 WatchSource:0}: Error finding container 697ff62ffea0afddffbe818dc94c7f363f95a3e19d630d4705fc1276dbcf2375: Status 404 returned error can't find the container with id 697ff62ffea0afddffbe818dc94c7f363f95a3e19d630d4705fc1276dbcf2375 Jul 12 18:29:08.424070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:08.424014 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64j9v" event={"ID":"6fb3be6c-d0ce-4185-8927-bc788b079877","Type":"ContainerStarted","Data":"697ff62ffea0afddffbe818dc94c7f363f95a3e19d630d4705fc1276dbcf2375"} Jul 12 18:29:09.430083 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:09.429977 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64j9v" event={"ID":"6fb3be6c-d0ce-4185-8927-bc788b079877","Type":"ContainerStarted","Data":"1d4831a933a50a0e0c2bd16c6bd36715024fa80648216d4128a36ad511f1ec51"} Jul 12 18:29:09.454341 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:09.454284 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-64j9v" podStartSLOduration=0.822748495 podStartE2EDuration="2.454267569s" totalImagesPullingTime="1.631519074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:29:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:29:09.452107186 +0000 UTC m=+240.651574251" watchObservedRunningTime="2026-07-12 18:29:09.454267569 +0000 UTC m=+240.653734598" Jul 12 18:29:10.780746 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.780703 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g"] Jul 12 18:29:10.784405 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.784379 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" Jul 12 18:29:10.786829 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.786805 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 18:29:10.786993 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.786869 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 18:29:10.786993 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.786874 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jb68p\"" Jul 12 18:29:10.793210 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.793183 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g"] Jul 12 18:29:10.839984 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.839950 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g\" (UID: \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" Jul 12 18:29:10.840147 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.840006 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g\" (UID: \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" Jul 12 18:29:10.840147 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.840034 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkqxm\" (UniqueName: \"kubernetes.io/projected/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-kube-api-access-qkqxm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g\" (UID: \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" Jul 12 18:29:10.941133 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.941100 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g\" (UID: \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" Jul 12 18:29:10.941300 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.941154 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g\" (UID: \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" Jul 12 18:29:10.941300 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.941174 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkqxm\" (UniqueName: \"kubernetes.io/projected/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-kube-api-access-qkqxm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g\" (UID: \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" Jul 12 18:29:10.941600 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.941577 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g\" (UID: \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" Jul 12 18:29:10.941642 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.941577 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g\" (UID: \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" Jul 12 18:29:10.950381 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:10.950344 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkqxm\" (UniqueName: \"kubernetes.io/projected/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-kube-api-access-qkqxm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g\" (UID: \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" Jul 12 18:29:11.094241 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:11.094147 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" Jul 12 18:29:11.231107 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:11.231066 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g"] Jul 12 18:29:11.234154 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:29:11.234124 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4f95f17_dfdf_42a2_a8ad_76dc7dcad73c.slice/crio-55f5c95a9861263fbd8d7fe028ed4e4cbface841588ee88885207221fe9e4311 WatchSource:0}: Error finding container 55f5c95a9861263fbd8d7fe028ed4e4cbface841588ee88885207221fe9e4311: Status 404 returned error can't find the container with id 55f5c95a9861263fbd8d7fe028ed4e4cbface841588ee88885207221fe9e4311 Jul 12 18:29:11.439435 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:11.439337 2585 generic.go:356] "Generic (PLEG): container finished" podID="b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c" containerID="1655936b5d5dbd3d0e793d11c45489a4ef1bead51a97a2c2eb8e2179569ba425" exitCode=0 Jul 12 18:29:11.439581 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:11.439443 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" event={"ID":"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c","Type":"ContainerDied","Data":"1655936b5d5dbd3d0e793d11c45489a4ef1bead51a97a2c2eb8e2179569ba425"} Jul 12 18:29:11.439581 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:11.439503 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" event={"ID":"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c","Type":"ContainerStarted","Data":"55f5c95a9861263fbd8d7fe028ed4e4cbface841588ee88885207221fe9e4311"} Jul 12 18:29:14.455043 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:14.454999 2585 generic.go:356] "Generic (PLEG): container finished" podID="b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c" containerID="d83f9d2abf4eb9a8d901a66cf65538374b7c3f017d828582b58ac36bfac2e7a8" exitCode=0 Jul 12 18:29:14.455427 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:14.455051 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" event={"ID":"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c","Type":"ContainerDied","Data":"d83f9d2abf4eb9a8d901a66cf65538374b7c3f017d828582b58ac36bfac2e7a8"} Jul 12 18:29:15.461077 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:15.461044 2585 generic.go:356] "Generic (PLEG): container finished" podID="b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c" containerID="e9ca5c6437eedf42bd0cf93cc2e1c5f25318a79e249686351d2fe593372c2f91" exitCode=0 Jul 12 18:29:15.461502 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:15.461135 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" event={"ID":"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c","Type":"ContainerDied","Data":"e9ca5c6437eedf42bd0cf93cc2e1c5f25318a79e249686351d2fe593372c2f91"} Jul 12 18:29:16.607391 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:16.607357 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" Jul 12 18:29:16.690352 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:16.690314 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-bundle\") pod \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\" (UID: \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\") " Jul 12 18:29:16.690524 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:16.690381 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-util\") pod \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\" (UID: \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\") " Jul 12 18:29:16.690524 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:16.690455 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkqxm\" (UniqueName: \"kubernetes.io/projected/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-kube-api-access-qkqxm\") pod \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\" (UID: \"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c\") " Jul 12 18:29:16.690744 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:16.690720 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-bundle" (OuterVolumeSpecName: "bundle") pod "b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c" (UID: "b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:29:16.692562 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:16.692532 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-kube-api-access-qkqxm" (OuterVolumeSpecName: "kube-api-access-qkqxm") pod "b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c" (UID: "b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c"). InnerVolumeSpecName "kube-api-access-qkqxm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:29:16.791419 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:16.791381 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-bundle\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:29:16.791419 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:16.791419 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkqxm\" (UniqueName: \"kubernetes.io/projected/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-kube-api-access-qkqxm\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:29:17.456117 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:17.456072 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-util" (OuterVolumeSpecName: "util") pod "b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c" (UID: "b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:29:17.471514 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:17.471485 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" Jul 12 18:29:17.471702 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:17.471485 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fs295g" event={"ID":"b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c","Type":"ContainerDied","Data":"55f5c95a9861263fbd8d7fe028ed4e4cbface841588ee88885207221fe9e4311"} Jul 12 18:29:17.471702 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:17.471610 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55f5c95a9861263fbd8d7fe028ed4e4cbface841588ee88885207221fe9e4311" Jul 12 18:29:17.498187 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:17.498155 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c-util\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:29:21.134173 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.134135 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-66j6r"] Jul 12 18:29:21.134626 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.134414 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c" containerName="util" Jul 12 18:29:21.134626 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.134425 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c" containerName="util" Jul 12 18:29:21.134626 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.134437 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c" containerName="extract" Jul 12 18:29:21.134626 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.134442 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c" containerName="extract" Jul 12 18:29:21.134626 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.134450 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c" containerName="pull" Jul 12 18:29:21.134626 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.134455 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c" containerName="pull" Jul 12 18:29:21.134626 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.134509 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4f95f17-dfdf-42a2-a8ad-76dc7dcad73c" containerName="extract" Jul 12 18:29:21.138512 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.138492 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-66j6r" Jul 12 18:29:21.141122 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.141097 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 12 18:29:21.141243 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.141095 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 12 18:29:21.142042 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.142024 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-lhgzv\"" Jul 12 18:29:21.149391 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.149361 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-66j6r"] Jul 12 18:29:21.226620 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.226575 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ddr4\" (UniqueName: \"kubernetes.io/projected/1c0620f9-b6aa-4571-859e-497759d11f5c-kube-api-access-9ddr4\") pod \"openshift-lws-operator-fd8ccff4c-66j6r\" (UID: \"1c0620f9-b6aa-4571-859e-497759d11f5c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-66j6r" Jul 12 18:29:21.226806 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.226629 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c0620f9-b6aa-4571-859e-497759d11f5c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-66j6r\" (UID: \"1c0620f9-b6aa-4571-859e-497759d11f5c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-66j6r" Jul 12 18:29:21.327122 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.327080 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ddr4\" (UniqueName: \"kubernetes.io/projected/1c0620f9-b6aa-4571-859e-497759d11f5c-kube-api-access-9ddr4\") pod \"openshift-lws-operator-fd8ccff4c-66j6r\" (UID: \"1c0620f9-b6aa-4571-859e-497759d11f5c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-66j6r" Jul 12 18:29:21.327122 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.327124 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c0620f9-b6aa-4571-859e-497759d11f5c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-66j6r\" (UID: \"1c0620f9-b6aa-4571-859e-497759d11f5c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-66j6r" Jul 12 18:29:21.327557 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.327538 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c0620f9-b6aa-4571-859e-497759d11f5c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-66j6r\" (UID: \"1c0620f9-b6aa-4571-859e-497759d11f5c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-66j6r" Jul 12 18:29:21.335619 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.335584 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ddr4\" (UniqueName: \"kubernetes.io/projected/1c0620f9-b6aa-4571-859e-497759d11f5c-kube-api-access-9ddr4\") pod \"openshift-lws-operator-fd8ccff4c-66j6r\" (UID: \"1c0620f9-b6aa-4571-859e-497759d11f5c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-66j6r" Jul 12 18:29:21.454881 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.454792 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-66j6r" Jul 12 18:29:21.587774 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:21.587592 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-66j6r"] Jul 12 18:29:21.593392 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:29:21.593342 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c0620f9_b6aa_4571_859e_497759d11f5c.slice/crio-94b9f4f0fc9878184b820972f5059d3962b279c98575d1112c4e70cba45a8ce5 WatchSource:0}: Error finding container 94b9f4f0fc9878184b820972f5059d3962b279c98575d1112c4e70cba45a8ce5: Status 404 returned error can't find the container with id 94b9f4f0fc9878184b820972f5059d3962b279c98575d1112c4e70cba45a8ce5 Jul 12 18:29:22.496038 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:22.495997 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-66j6r" event={"ID":"1c0620f9-b6aa-4571-859e-497759d11f5c","Type":"ContainerStarted","Data":"94b9f4f0fc9878184b820972f5059d3962b279c98575d1112c4e70cba45a8ce5"} Jul 12 18:29:23.509651 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.509600 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-66j6r" event={"ID":"1c0620f9-b6aa-4571-859e-497759d11f5c","Type":"ContainerStarted","Data":"75ba5f4598d4959ac19787d9fc00bf0cb6fae9bfe7043e338fc60210fe91166b"} Jul 12 18:29:23.525625 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.525565 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-66j6r" podStartSLOduration=0.707646263 podStartE2EDuration="2.525548036s" totalImagesPullingTime="1.817901773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:29:23.525301276 +0000 UTC m=+254.724768306" watchObservedRunningTime="2026-07-12 18:29:23.525548036 +0000 UTC m=+254.725015066" Jul 12 18:29:23.638827 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.638781 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qggrx"] Jul 12 18:29:23.642250 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.642221 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qggrx" Jul 12 18:29:23.644612 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.644580 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 12 18:29:23.644775 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.644702 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-5wbtc\"" Jul 12 18:29:23.644775 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.644731 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 12 18:29:23.651738 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.651710 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qggrx"] Jul 12 18:29:23.744226 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.744130 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzqk9\" (UniqueName: \"kubernetes.io/projected/4144d918-c253-49be-883e-8a8a547edbc6-kube-api-access-pzqk9\") pod \"cert-manager-55b4d954d5-qggrx\" (UID: \"4144d918-c253-49be-883e-8a8a547edbc6\") " pod="cert-manager/cert-manager-55b4d954d5-qggrx" Jul 12 18:29:23.744226 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.744190 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4144d918-c253-49be-883e-8a8a547edbc6-bound-sa-token\") pod \"cert-manager-55b4d954d5-qggrx\" (UID: \"4144d918-c253-49be-883e-8a8a547edbc6\") " pod="cert-manager/cert-manager-55b4d954d5-qggrx" Jul 12 18:29:23.845060 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.845019 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4144d918-c253-49be-883e-8a8a547edbc6-bound-sa-token\") pod \"cert-manager-55b4d954d5-qggrx\" (UID: \"4144d918-c253-49be-883e-8a8a547edbc6\") " pod="cert-manager/cert-manager-55b4d954d5-qggrx" Jul 12 18:29:23.845207 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.845120 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzqk9\" (UniqueName: \"kubernetes.io/projected/4144d918-c253-49be-883e-8a8a547edbc6-kube-api-access-pzqk9\") pod \"cert-manager-55b4d954d5-qggrx\" (UID: \"4144d918-c253-49be-883e-8a8a547edbc6\") " pod="cert-manager/cert-manager-55b4d954d5-qggrx" Jul 12 18:29:23.853400 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.853370 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4144d918-c253-49be-883e-8a8a547edbc6-bound-sa-token\") pod \"cert-manager-55b4d954d5-qggrx\" (UID: \"4144d918-c253-49be-883e-8a8a547edbc6\") " pod="cert-manager/cert-manager-55b4d954d5-qggrx" Jul 12 18:29:23.853523 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.853475 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzqk9\" (UniqueName: \"kubernetes.io/projected/4144d918-c253-49be-883e-8a8a547edbc6-kube-api-access-pzqk9\") pod \"cert-manager-55b4d954d5-qggrx\" (UID: \"4144d918-c253-49be-883e-8a8a547edbc6\") " pod="cert-manager/cert-manager-55b4d954d5-qggrx" Jul 12 18:29:23.952276 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:23.952238 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qggrx" Jul 12 18:29:24.096975 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:24.096924 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qggrx"] Jul 12 18:29:24.099806 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:29:24.099774 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4144d918_c253_49be_883e_8a8a547edbc6.slice/crio-9917cf64d32a6fc07364a078fcb4007dd03eacc951b4733c0cd4940fd2fd8a88 WatchSource:0}: Error finding container 9917cf64d32a6fc07364a078fcb4007dd03eacc951b4733c0cd4940fd2fd8a88: Status 404 returned error can't find the container with id 9917cf64d32a6fc07364a078fcb4007dd03eacc951b4733c0cd4940fd2fd8a88 Jul 12 18:29:24.514435 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:24.514395 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qggrx" event={"ID":"4144d918-c253-49be-883e-8a8a547edbc6","Type":"ContainerStarted","Data":"9917cf64d32a6fc07364a078fcb4007dd03eacc951b4733c0cd4940fd2fd8a88"} Jul 12 18:29:26.189668 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.189634 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6"] Jul 12 18:29:26.209325 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.209270 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6"] Jul 12 18:29:26.209493 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.209396 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" Jul 12 18:29:26.212050 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.212020 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 18:29:26.212795 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.212765 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jb68p\"" Jul 12 18:29:26.213009 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.212795 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 18:29:26.264997 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.264956 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcfpg\" (UniqueName: \"kubernetes.io/projected/2354f0f3-69cf-4424-9216-ab1493eaec32-kube-api-access-pcfpg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6\" (UID: \"2354f0f3-69cf-4424-9216-ab1493eaec32\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" Jul 12 18:29:26.265168 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.265100 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2354f0f3-69cf-4424-9216-ab1493eaec32-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6\" (UID: \"2354f0f3-69cf-4424-9216-ab1493eaec32\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" Jul 12 18:29:26.265168 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.265147 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2354f0f3-69cf-4424-9216-ab1493eaec32-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6\" (UID: \"2354f0f3-69cf-4424-9216-ab1493eaec32\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" Jul 12 18:29:26.365807 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.365750 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcfpg\" (UniqueName: \"kubernetes.io/projected/2354f0f3-69cf-4424-9216-ab1493eaec32-kube-api-access-pcfpg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6\" (UID: \"2354f0f3-69cf-4424-9216-ab1493eaec32\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" Jul 12 18:29:26.366034 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.365872 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2354f0f3-69cf-4424-9216-ab1493eaec32-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6\" (UID: \"2354f0f3-69cf-4424-9216-ab1493eaec32\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" Jul 12 18:29:26.366034 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.365905 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2354f0f3-69cf-4424-9216-ab1493eaec32-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6\" (UID: \"2354f0f3-69cf-4424-9216-ab1493eaec32\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" Jul 12 18:29:26.366452 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.366430 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2354f0f3-69cf-4424-9216-ab1493eaec32-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6\" (UID: \"2354f0f3-69cf-4424-9216-ab1493eaec32\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" Jul 12 18:29:26.366541 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.366520 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2354f0f3-69cf-4424-9216-ab1493eaec32-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6\" (UID: \"2354f0f3-69cf-4424-9216-ab1493eaec32\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" Jul 12 18:29:26.375325 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.375293 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcfpg\" (UniqueName: \"kubernetes.io/projected/2354f0f3-69cf-4424-9216-ab1493eaec32-kube-api-access-pcfpg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6\" (UID: \"2354f0f3-69cf-4424-9216-ab1493eaec32\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" Jul 12 18:29:26.521690 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.521657 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" Jul 12 18:29:26.882014 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:26.881807 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6"] Jul 12 18:29:26.884554 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:29:26.884524 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2354f0f3_69cf_4424_9216_ab1493eaec32.slice/crio-2d983b27d0369cda0145fe00c4326dcfa200242dbb206a4e67530d3249097314 WatchSource:0}: Error finding container 2d983b27d0369cda0145fe00c4326dcfa200242dbb206a4e67530d3249097314: Status 404 returned error can't find the container with id 2d983b27d0369cda0145fe00c4326dcfa200242dbb206a4e67530d3249097314 Jul 12 18:29:27.529946 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:27.529903 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qggrx" event={"ID":"4144d918-c253-49be-883e-8a8a547edbc6","Type":"ContainerStarted","Data":"b69c339ef3a5978421efa214094c888f58c018583700b358183141d01b8dee3b"} Jul 12 18:29:27.531362 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:27.531341 2585 generic.go:356] "Generic (PLEG): container finished" podID="2354f0f3-69cf-4424-9216-ab1493eaec32" containerID="ad9084b55b6f5fdb08ec37be94337c80067e1a9993410c01f1abbf8ce895bc07" exitCode=0 Jul 12 18:29:27.531438 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:27.531420 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" event={"ID":"2354f0f3-69cf-4424-9216-ab1493eaec32","Type":"ContainerDied","Data":"ad9084b55b6f5fdb08ec37be94337c80067e1a9993410c01f1abbf8ce895bc07"} Jul 12 18:29:27.531478 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:27.531466 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" event={"ID":"2354f0f3-69cf-4424-9216-ab1493eaec32","Type":"ContainerStarted","Data":"2d983b27d0369cda0145fe00c4326dcfa200242dbb206a4e67530d3249097314"} Jul 12 18:29:27.546846 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:27.546781 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-qggrx" podStartSLOduration=1.447863757 podStartE2EDuration="4.546763269s" totalImagesPullingTime="3.098899512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:29:27.544580217 +0000 UTC m=+258.744047248" watchObservedRunningTime="2026-07-12 18:29:27.546763269 +0000 UTC m=+258.746230299" Jul 12 18:29:28.537768 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:28.537722 2585 generic.go:356] "Generic (PLEG): container finished" podID="2354f0f3-69cf-4424-9216-ab1493eaec32" containerID="672c6262799bdf220c5a4fd851b2a1e55e572f99e26447c33f0c66a7fc7258dc" exitCode=0 Jul 12 18:29:28.538158 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:28.537802 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" event={"ID":"2354f0f3-69cf-4424-9216-ab1493eaec32","Type":"ContainerDied","Data":"672c6262799bdf220c5a4fd851b2a1e55e572f99e26447c33f0c66a7fc7258dc"} Jul 12 18:29:29.544235 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:29.544194 2585 generic.go:356] "Generic (PLEG): container finished" podID="2354f0f3-69cf-4424-9216-ab1493eaec32" containerID="b6ee29c37e9528043050d472e8055f6bd5cfeaa47d1a813a5fc3be90e41f3e8b" exitCode=0 Jul 12 18:29:29.544593 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:29.544255 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" event={"ID":"2354f0f3-69cf-4424-9216-ab1493eaec32","Type":"ContainerDied","Data":"b6ee29c37e9528043050d472e8055f6bd5cfeaa47d1a813a5fc3be90e41f3e8b"} Jul 12 18:29:30.674654 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:30.674627 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" Jul 12 18:29:30.802486 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:30.802385 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcfpg\" (UniqueName: \"kubernetes.io/projected/2354f0f3-69cf-4424-9216-ab1493eaec32-kube-api-access-pcfpg\") pod \"2354f0f3-69cf-4424-9216-ab1493eaec32\" (UID: \"2354f0f3-69cf-4424-9216-ab1493eaec32\") " Jul 12 18:29:30.802486 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:30.802427 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2354f0f3-69cf-4424-9216-ab1493eaec32-util\") pod \"2354f0f3-69cf-4424-9216-ab1493eaec32\" (UID: \"2354f0f3-69cf-4424-9216-ab1493eaec32\") " Jul 12 18:29:30.802727 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:30.802492 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2354f0f3-69cf-4424-9216-ab1493eaec32-bundle\") pod \"2354f0f3-69cf-4424-9216-ab1493eaec32\" (UID: \"2354f0f3-69cf-4424-9216-ab1493eaec32\") " Jul 12 18:29:30.803271 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:30.803245 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2354f0f3-69cf-4424-9216-ab1493eaec32-bundle" (OuterVolumeSpecName: "bundle") pod "2354f0f3-69cf-4424-9216-ab1493eaec32" (UID: "2354f0f3-69cf-4424-9216-ab1493eaec32"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:29:30.804687 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:30.804659 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2354f0f3-69cf-4424-9216-ab1493eaec32-kube-api-access-pcfpg" (OuterVolumeSpecName: "kube-api-access-pcfpg") pod "2354f0f3-69cf-4424-9216-ab1493eaec32" (UID: "2354f0f3-69cf-4424-9216-ab1493eaec32"). InnerVolumeSpecName "kube-api-access-pcfpg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:29:30.808950 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:30.808893 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2354f0f3-69cf-4424-9216-ab1493eaec32-util" (OuterVolumeSpecName: "util") pod "2354f0f3-69cf-4424-9216-ab1493eaec32" (UID: "2354f0f3-69cf-4424-9216-ab1493eaec32"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:29:30.903512 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:30.903473 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2354f0f3-69cf-4424-9216-ab1493eaec32-bundle\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:29:30.903512 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:30.903506 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcfpg\" (UniqueName: \"kubernetes.io/projected/2354f0f3-69cf-4424-9216-ab1493eaec32-kube-api-access-pcfpg\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:29:30.903512 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:30.903521 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2354f0f3-69cf-4424-9216-ab1493eaec32-util\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:29:31.555428 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:31.555382 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" event={"ID":"2354f0f3-69cf-4424-9216-ab1493eaec32","Type":"ContainerDied","Data":"2d983b27d0369cda0145fe00c4326dcfa200242dbb206a4e67530d3249097314"} Jul 12 18:29:31.555720 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:31.555450 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d983b27d0369cda0145fe00c4326dcfa200242dbb206a4e67530d3249097314" Jul 12 18:29:31.555720 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:31.555396 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7j6b6" Jul 12 18:29:42.763451 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.763415 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59"] Jul 12 18:29:42.763903 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.763720 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2354f0f3-69cf-4424-9216-ab1493eaec32" containerName="extract" Jul 12 18:29:42.763903 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.763732 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2354f0f3-69cf-4424-9216-ab1493eaec32" containerName="extract" Jul 12 18:29:42.763903 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.763745 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2354f0f3-69cf-4424-9216-ab1493eaec32" containerName="util" Jul 12 18:29:42.763903 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.763750 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2354f0f3-69cf-4424-9216-ab1493eaec32" containerName="util" Jul 12 18:29:42.763903 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.763762 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2354f0f3-69cf-4424-9216-ab1493eaec32" containerName="pull" Jul 12 18:29:42.763903 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.763767 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2354f0f3-69cf-4424-9216-ab1493eaec32" containerName="pull" Jul 12 18:29:42.763903 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.763815 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2354f0f3-69cf-4424-9216-ab1493eaec32" containerName="extract" Jul 12 18:29:42.768122 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.768094 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:42.772103 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.772077 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 12 18:29:42.772257 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.772122 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 12 18:29:42.772257 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.772168 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 12 18:29:42.772257 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.772129 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 12 18:29:42.772257 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.772199 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-d6g59\"" Jul 12 18:29:42.782109 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.782079 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59"] Jul 12 18:29:42.902051 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.902009 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3d6039e0-174a-4524-ba46-500c1604c808-manifests\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:42.902051 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.902055 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d6039e0-174a-4524-ba46-500c1604c808-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:42.902288 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.902078 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3d6039e0-174a-4524-ba46-500c1604c808-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:42.902288 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.902130 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jphrh\" (UniqueName: \"kubernetes.io/projected/3d6039e0-174a-4524-ba46-500c1604c808-kube-api-access-jphrh\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:42.902288 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:42.902192 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d6039e0-174a-4524-ba46-500c1604c808-tmp\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:43.003366 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:43.003323 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3d6039e0-174a-4524-ba46-500c1604c808-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:43.003366 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:43.003372 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jphrh\" (UniqueName: \"kubernetes.io/projected/3d6039e0-174a-4524-ba46-500c1604c808-kube-api-access-jphrh\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:43.003630 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:43.003406 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d6039e0-174a-4524-ba46-500c1604c808-tmp\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:43.003630 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:43.003465 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3d6039e0-174a-4524-ba46-500c1604c808-manifests\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:43.003630 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:43.003495 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d6039e0-174a-4524-ba46-500c1604c808-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:43.003975 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:43.003917 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3d6039e0-174a-4524-ba46-500c1604c808-manifests\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:43.005788 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:43.005757 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d6039e0-174a-4524-ba46-500c1604c808-tmp\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:43.005887 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:43.005857 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d6039e0-174a-4524-ba46-500c1604c808-webhook-cert\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:43.005954 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:43.005899 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3d6039e0-174a-4524-ba46-500c1604c808-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:43.011800 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:43.011778 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jphrh\" (UniqueName: \"kubernetes.io/projected/3d6039e0-174a-4524-ba46-500c1604c808-kube-api-access-jphrh\") pod \"opendatahub-operator-controller-manager-59f88dd4c8-l6j59\" (UID: \"3d6039e0-174a-4524-ba46-500c1604c808\") " pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:43.079689 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:43.079657 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:43.223863 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:43.223834 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59"] Jul 12 18:29:43.226953 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:29:43.226898 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d6039e0_174a_4524_ba46_500c1604c808.slice/crio-0173e48012fa65668f2eff1150c63012b58ef6a6f448834f684996a8407bc616 WatchSource:0}: Error finding container 0173e48012fa65668f2eff1150c63012b58ef6a6f448834f684996a8407bc616: Status 404 returned error can't find the container with id 0173e48012fa65668f2eff1150c63012b58ef6a6f448834f684996a8407bc616 Jul 12 18:29:43.610472 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:43.610436 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" event={"ID":"3d6039e0-174a-4524-ba46-500c1604c808","Type":"ContainerStarted","Data":"0173e48012fa65668f2eff1150c63012b58ef6a6f448834f684996a8407bc616"} Jul 12 18:29:46.629299 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:46.629258 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" event={"ID":"3d6039e0-174a-4524-ba46-500c1604c808","Type":"ContainerStarted","Data":"f9323435b385a765840c1cd4fd5a089be8a551f03d8b6a11c995debd472e35f1"} Jul 12 18:29:47.635565 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:47.635470 2585 generic.go:356] "Generic (PLEG): container finished" podID="3d6039e0-174a-4524-ba46-500c1604c808" containerID="f9323435b385a765840c1cd4fd5a089be8a551f03d8b6a11c995debd472e35f1" exitCode=0 Jul 12 18:29:47.635920 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:47.635559 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" event={"ID":"3d6039e0-174a-4524-ba46-500c1604c808","Type":"ContainerDied","Data":"f9323435b385a765840c1cd4fd5a089be8a551f03d8b6a11c995debd472e35f1"} Jul 12 18:29:48.642583 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:48.642547 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" event={"ID":"3d6039e0-174a-4524-ba46-500c1604c808","Type":"ContainerStarted","Data":"940ca421d0b91257f7a21fc5d767cb2c8bc2384ef5f93e7fd27c03b34190f4a2"} Jul 12 18:29:48.643106 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:48.642658 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:29:48.666200 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:48.666148 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" podStartSLOduration=2.081929376 podStartE2EDuration="6.666126252s" totalImagesPullingTime="3.486083109s" totalInitContainerRuntime="1.098113767s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:29:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:29:48.664221116 +0000 UTC m=+279.863688146" watchObservedRunningTime="2026-07-12 18:29:48.666126252 +0000 UTC m=+279.865593283" Jul 12 18:29:59.649529 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:29:59.649488 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59f88dd4c8-l6j59" Jul 12 18:30:02.887640 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:02.887600 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m"] Jul 12 18:30:02.890650 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:02.890628 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" Jul 12 18:30:02.893136 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:02.893108 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 12 18:30:02.893258 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:02.893109 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 12 18:30:02.893258 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:02.893117 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gd7c4\"" Jul 12 18:30:02.904767 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:02.904735 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m"] Jul 12 18:30:03.074241 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.074194 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6jfw\" (UniqueName: \"kubernetes.io/projected/f146b5fb-f81f-4a5d-a9b6-82b800bea9ca-kube-api-access-x6jfw\") pod \"kube-auth-proxy-5dd95989b6-86z8m\" (UID: \"f146b5fb-f81f-4a5d-a9b6-82b800bea9ca\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" Jul 12 18:30:03.074428 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.074258 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f146b5fb-f81f-4a5d-a9b6-82b800bea9ca-tmp\") pod \"kube-auth-proxy-5dd95989b6-86z8m\" (UID: \"f146b5fb-f81f-4a5d-a9b6-82b800bea9ca\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" Jul 12 18:30:03.074428 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.074377 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f146b5fb-f81f-4a5d-a9b6-82b800bea9ca-tls-certs\") pod \"kube-auth-proxy-5dd95989b6-86z8m\" (UID: \"f146b5fb-f81f-4a5d-a9b6-82b800bea9ca\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" Jul 12 18:30:03.100807 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.100767 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6f7h9"] Jul 12 18:30:03.104094 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.104073 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6f7h9" Jul 12 18:30:03.106419 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.106393 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 12 18:30:03.106589 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.106569 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-plg5c\"" Jul 12 18:30:03.115337 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.115309 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6f7h9"] Jul 12 18:30:03.175506 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.175418 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f146b5fb-f81f-4a5d-a9b6-82b800bea9ca-tls-certs\") pod \"kube-auth-proxy-5dd95989b6-86z8m\" (UID: \"f146b5fb-f81f-4a5d-a9b6-82b800bea9ca\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" Jul 12 18:30:03.175506 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.175461 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6jfw\" (UniqueName: \"kubernetes.io/projected/f146b5fb-f81f-4a5d-a9b6-82b800bea9ca-kube-api-access-x6jfw\") pod \"kube-auth-proxy-5dd95989b6-86z8m\" (UID: \"f146b5fb-f81f-4a5d-a9b6-82b800bea9ca\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" Jul 12 18:30:03.175506 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.175500 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f146b5fb-f81f-4a5d-a9b6-82b800bea9ca-tmp\") pod \"kube-auth-proxy-5dd95989b6-86z8m\" (UID: \"f146b5fb-f81f-4a5d-a9b6-82b800bea9ca\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" Jul 12 18:30:03.175734 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:30:03.175604 2585 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 12 18:30:03.175734 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:30:03.175687 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f146b5fb-f81f-4a5d-a9b6-82b800bea9ca-tls-certs podName:f146b5fb-f81f-4a5d-a9b6-82b800bea9ca nodeName:}" failed. No retries permitted until 2026-07-12 18:30:03.675664787 +0000 UTC m=+294.875131802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f146b5fb-f81f-4a5d-a9b6-82b800bea9ca-tls-certs") pod "kube-auth-proxy-5dd95989b6-86z8m" (UID: "f146b5fb-f81f-4a5d-a9b6-82b800bea9ca") : secret "kube-auth-proxy-tls" not found Jul 12 18:30:03.178135 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.178096 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f146b5fb-f81f-4a5d-a9b6-82b800bea9ca-tmp\") pod \"kube-auth-proxy-5dd95989b6-86z8m\" (UID: \"f146b5fb-f81f-4a5d-a9b6-82b800bea9ca\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" Jul 12 18:30:03.186578 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.186555 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6jfw\" (UniqueName: \"kubernetes.io/projected/f146b5fb-f81f-4a5d-a9b6-82b800bea9ca-kube-api-access-x6jfw\") pod \"kube-auth-proxy-5dd95989b6-86z8m\" (UID: \"f146b5fb-f81f-4a5d-a9b6-82b800bea9ca\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" Jul 12 18:30:03.276577 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.276528 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smxk4\" (UniqueName: \"kubernetes.io/projected/59e7df2c-c8b5-4163-9bc7-baa9125014df-kube-api-access-smxk4\") pod \"odh-model-controller-7998c67dc4-6f7h9\" (UID: \"59e7df2c-c8b5-4163-9bc7-baa9125014df\") " pod="opendatahub/odh-model-controller-7998c67dc4-6f7h9" Jul 12 18:30:03.276757 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.276686 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59e7df2c-c8b5-4163-9bc7-baa9125014df-cert\") pod \"odh-model-controller-7998c67dc4-6f7h9\" (UID: \"59e7df2c-c8b5-4163-9bc7-baa9125014df\") " pod="opendatahub/odh-model-controller-7998c67dc4-6f7h9" Jul 12 18:30:03.378059 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.378015 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59e7df2c-c8b5-4163-9bc7-baa9125014df-cert\") pod \"odh-model-controller-7998c67dc4-6f7h9\" (UID: \"59e7df2c-c8b5-4163-9bc7-baa9125014df\") " pod="opendatahub/odh-model-controller-7998c67dc4-6f7h9" Jul 12 18:30:03.378059 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.378065 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smxk4\" (UniqueName: \"kubernetes.io/projected/59e7df2c-c8b5-4163-9bc7-baa9125014df-kube-api-access-smxk4\") pod \"odh-model-controller-7998c67dc4-6f7h9\" (UID: \"59e7df2c-c8b5-4163-9bc7-baa9125014df\") " pod="opendatahub/odh-model-controller-7998c67dc4-6f7h9" Jul 12 18:30:03.378327 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:30:03.378199 2585 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 12 18:30:03.378327 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:30:03.378267 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59e7df2c-c8b5-4163-9bc7-baa9125014df-cert podName:59e7df2c-c8b5-4163-9bc7-baa9125014df nodeName:}" failed. No retries permitted until 2026-07-12 18:30:03.878252192 +0000 UTC m=+295.077719203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/59e7df2c-c8b5-4163-9bc7-baa9125014df-cert") pod "odh-model-controller-7998c67dc4-6f7h9" (UID: "59e7df2c-c8b5-4163-9bc7-baa9125014df") : secret "odh-model-controller-webhook-cert" not found Jul 12 18:30:03.392958 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.392895 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smxk4\" (UniqueName: \"kubernetes.io/projected/59e7df2c-c8b5-4163-9bc7-baa9125014df-kube-api-access-smxk4\") pod \"odh-model-controller-7998c67dc4-6f7h9\" (UID: \"59e7df2c-c8b5-4163-9bc7-baa9125014df\") " pod="opendatahub/odh-model-controller-7998c67dc4-6f7h9" Jul 12 18:30:03.680722 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.680677 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f146b5fb-f81f-4a5d-a9b6-82b800bea9ca-tls-certs\") pod \"kube-auth-proxy-5dd95989b6-86z8m\" (UID: \"f146b5fb-f81f-4a5d-a9b6-82b800bea9ca\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" Jul 12 18:30:03.683169 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.683147 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f146b5fb-f81f-4a5d-a9b6-82b800bea9ca-tls-certs\") pod \"kube-auth-proxy-5dd95989b6-86z8m\" (UID: \"f146b5fb-f81f-4a5d-a9b6-82b800bea9ca\") " pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" Jul 12 18:30:03.800212 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.800172 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" Jul 12 18:30:03.882648 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.882607 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59e7df2c-c8b5-4163-9bc7-baa9125014df-cert\") pod \"odh-model-controller-7998c67dc4-6f7h9\" (UID: \"59e7df2c-c8b5-4163-9bc7-baa9125014df\") " pod="opendatahub/odh-model-controller-7998c67dc4-6f7h9" Jul 12 18:30:03.885551 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.885482 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59e7df2c-c8b5-4163-9bc7-baa9125014df-cert\") pod \"odh-model-controller-7998c67dc4-6f7h9\" (UID: \"59e7df2c-c8b5-4163-9bc7-baa9125014df\") " pod="opendatahub/odh-model-controller-7998c67dc4-6f7h9" Jul 12 18:30:03.937340 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:03.937309 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m"] Jul 12 18:30:03.939476 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:30:03.939452 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf146b5fb_f81f_4a5d_a9b6_82b800bea9ca.slice/crio-cadf586d51930d3d3445519ae165e39d85e86df6692ec6f6c6f1ce2f8ddb046b WatchSource:0}: Error finding container cadf586d51930d3d3445519ae165e39d85e86df6692ec6f6c6f1ce2f8ddb046b: Status 404 returned error can't find the container with id cadf586d51930d3d3445519ae165e39d85e86df6692ec6f6c6f1ce2f8ddb046b Jul 12 18:30:04.015168 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.015133 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6f7h9" Jul 12 18:30:04.125773 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.125727 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-l5nxv"] Jul 12 18:30:04.129396 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.129371 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-l5nxv" Jul 12 18:30:04.131911 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.131882 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 12 18:30:04.141632 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.141587 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-l5nxv"] Jul 12 18:30:04.164377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.164321 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6f7h9"] Jul 12 18:30:04.166944 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:30:04.166906 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59e7df2c_c8b5_4163_9bc7_baa9125014df.slice/crio-5cf40fe0a87f9fb816764a2374aeec748e7f1b69cf9e667e7fa6ef90e2e9f6d2 WatchSource:0}: Error finding container 5cf40fe0a87f9fb816764a2374aeec748e7f1b69cf9e667e7fa6ef90e2e9f6d2: Status 404 returned error can't find the container with id 5cf40fe0a87f9fb816764a2374aeec748e7f1b69cf9e667e7fa6ef90e2e9f6d2 Jul 12 18:30:04.286313 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.286274 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b96066fe-ee3b-40a9-80fb-e1fd4bbeac68-tls-certs\") pod \"model-serving-api-54d89bd5fc-l5nxv\" (UID: \"b96066fe-ee3b-40a9-80fb-e1fd4bbeac68\") " pod="opendatahub/model-serving-api-54d89bd5fc-l5nxv" Jul 12 18:30:04.286493 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.286319 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2zm7\" (UniqueName: \"kubernetes.io/projected/b96066fe-ee3b-40a9-80fb-e1fd4bbeac68-kube-api-access-l2zm7\") pod \"model-serving-api-54d89bd5fc-l5nxv\" (UID: \"b96066fe-ee3b-40a9-80fb-e1fd4bbeac68\") " pod="opendatahub/model-serving-api-54d89bd5fc-l5nxv" Jul 12 18:30:04.387743 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.387706 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b96066fe-ee3b-40a9-80fb-e1fd4bbeac68-tls-certs\") pod \"model-serving-api-54d89bd5fc-l5nxv\" (UID: \"b96066fe-ee3b-40a9-80fb-e1fd4bbeac68\") " pod="opendatahub/model-serving-api-54d89bd5fc-l5nxv" Jul 12 18:30:04.387743 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.387748 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2zm7\" (UniqueName: \"kubernetes.io/projected/b96066fe-ee3b-40a9-80fb-e1fd4bbeac68-kube-api-access-l2zm7\") pod \"model-serving-api-54d89bd5fc-l5nxv\" (UID: \"b96066fe-ee3b-40a9-80fb-e1fd4bbeac68\") " pod="opendatahub/model-serving-api-54d89bd5fc-l5nxv" Jul 12 18:30:04.390142 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.390112 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b96066fe-ee3b-40a9-80fb-e1fd4bbeac68-tls-certs\") pod \"model-serving-api-54d89bd5fc-l5nxv\" (UID: \"b96066fe-ee3b-40a9-80fb-e1fd4bbeac68\") " pod="opendatahub/model-serving-api-54d89bd5fc-l5nxv" Jul 12 18:30:04.403899 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.403866 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2zm7\" (UniqueName: \"kubernetes.io/projected/b96066fe-ee3b-40a9-80fb-e1fd4bbeac68-kube-api-access-l2zm7\") pod \"model-serving-api-54d89bd5fc-l5nxv\" (UID: \"b96066fe-ee3b-40a9-80fb-e1fd4bbeac68\") " pod="opendatahub/model-serving-api-54d89bd5fc-l5nxv" Jul 12 18:30:04.443902 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.443856 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-l5nxv" Jul 12 18:30:04.580729 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.580700 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-l5nxv"] Jul 12 18:30:04.582861 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:30:04.582829 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb96066fe_ee3b_40a9_80fb_e1fd4bbeac68.slice/crio-4656d68b89228759b87157ceec1ac6e819c784c20323b1757d7c8e1c9e96460b WatchSource:0}: Error finding container 4656d68b89228759b87157ceec1ac6e819c784c20323b1757d7c8e1c9e96460b: Status 404 returned error can't find the container with id 4656d68b89228759b87157ceec1ac6e819c784c20323b1757d7c8e1c9e96460b Jul 12 18:30:04.717319 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.717282 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" event={"ID":"f146b5fb-f81f-4a5d-a9b6-82b800bea9ca","Type":"ContainerStarted","Data":"cadf586d51930d3d3445519ae165e39d85e86df6692ec6f6c6f1ce2f8ddb046b"} Jul 12 18:30:04.718822 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.718787 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-l5nxv" event={"ID":"b96066fe-ee3b-40a9-80fb-e1fd4bbeac68","Type":"ContainerStarted","Data":"4656d68b89228759b87157ceec1ac6e819c784c20323b1757d7c8e1c9e96460b"} Jul 12 18:30:04.720249 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:04.720212 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6f7h9" event={"ID":"59e7df2c-c8b5-4163-9bc7-baa9125014df","Type":"ContainerStarted","Data":"5cf40fe0a87f9fb816764a2374aeec748e7f1b69cf9e667e7fa6ef90e2e9f6d2"} Jul 12 18:30:08.744796 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:08.744757 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" event={"ID":"f146b5fb-f81f-4a5d-a9b6-82b800bea9ca","Type":"ContainerStarted","Data":"b903a8d5651dd56646951124ecc863f84db1912ac9fd3f22fdea813fa60ca979"} Jul 12 18:30:08.746410 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:08.746383 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-l5nxv" event={"ID":"b96066fe-ee3b-40a9-80fb-e1fd4bbeac68","Type":"ContainerStarted","Data":"19c78236b7503b127a1bf6938e16f4e7e8d270f38f3a3cac479e2b4feb2034a6"} Jul 12 18:30:08.746527 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:08.746482 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-l5nxv" Jul 12 18:30:08.747828 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:08.747805 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6f7h9" event={"ID":"59e7df2c-c8b5-4163-9bc7-baa9125014df","Type":"ContainerStarted","Data":"09d21bbbe9d2e40d21b987f6f117598bb114448d11ff71e9daf0faca7439518a"} Jul 12 18:30:08.747953 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:08.747944 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-6f7h9" Jul 12 18:30:08.784723 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:08.784661 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5dd95989b6-86z8m" podStartSLOduration=2.665034314 podStartE2EDuration="6.784645218s" totalImagesPullingTime="4.119610904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:30:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:30:08.783754405 +0000 UTC m=+299.983221448" watchObservedRunningTime="2026-07-12 18:30:08.784645218 +0000 UTC m=+299.984112248" Jul 12 18:30:08.802045 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:08.801984 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-6f7h9" podStartSLOduration=1.895043566 podStartE2EDuration="5.801965929s" totalImagesPullingTime="3.906922363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:30:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:30:08.800440639 +0000 UTC m=+299.999907681" watchObservedRunningTime="2026-07-12 18:30:08.801965929 +0000 UTC m=+300.001432959" Jul 12 18:30:08.819919 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:08.819860 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-l5nxv" podStartSLOduration=1.339262265 podStartE2EDuration="4.819839353s" totalImagesPullingTime="3.480577088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:30:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:30:08.818679512 +0000 UTC m=+300.018146549" watchObservedRunningTime="2026-07-12 18:30:08.819839353 +0000 UTC m=+300.019306383" Jul 12 18:30:09.244527 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:09.244500 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:30:09.248650 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:09.248616 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:30:09.258606 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:09.258574 2585 kubelet.go:1648] "Image garbage collection succeeded" Jul 12 18:30:10.847686 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.847649 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w"] Jul 12 18:30:10.851116 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.851095 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:10.853850 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.853828 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-6x5bn\"" Jul 12 18:30:10.853994 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.853901 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 12 18:30:10.863662 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.863637 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w"] Jul 12 18:30:10.943765 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.943722 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:10.943765 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.943779 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:10.944129 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.943971 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:10.944129 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.944025 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-452p9\" (UniqueName: \"kubernetes.io/projected/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-kube-api-access-452p9\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:10.944129 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.944054 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:10.944129 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.944099 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:10.944129 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.944123 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:10.944347 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.944145 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:10.944347 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.944171 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:10.946597 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.946552 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc"] Jul 12 18:30:10.950019 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.949996 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:10.953123 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.953101 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 12 18:30:10.953277 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.953256 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 12 18:30:10.953538 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.953520 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jfm46\"" Jul 12 18:30:10.953736 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.953722 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 12 18:30:10.963561 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:10.963533 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc"] Jul 12 18:30:11.045249 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045211 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.045443 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045257 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3b2bba9-377f-4663-9c77-95f1a1e8910c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.045443 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045284 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f3b2bba9-377f-4663-9c77-95f1a1e8910c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.045443 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045311 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.045443 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045369 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.045443 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045412 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.045443 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045442 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3b2bba9-377f-4663-9c77-95f1a1e8910c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.045728 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045493 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.045728 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045522 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f3b2bba9-377f-4663-9c77-95f1a1e8910c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.045728 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045549 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-889fz\" (UniqueName: \"kubernetes.io/projected/f3b2bba9-377f-4663-9c77-95f1a1e8910c-kube-api-access-889fz\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.045728 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045584 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f3b2bba9-377f-4663-9c77-95f1a1e8910c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.045728 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045657 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.045728 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045665 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.045728 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045716 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f3b2bba9-377f-4663-9c77-95f1a1e8910c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.046144 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045745 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.046144 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045819 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-452p9\" (UniqueName: \"kubernetes.io/projected/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-kube-api-access-452p9\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.046144 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045850 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.046144 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045859 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.046144 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.045914 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.046144 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.046095 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.046434 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.046314 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.047903 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.047880 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.048232 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.048216 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.056446 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.056399 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-452p9\" (UniqueName: \"kubernetes.io/projected/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-kube-api-access-452p9\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.057142 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.057121 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e80edd8c-d4a2-4bda-9ce6-029183c3ede1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mdl9w\" (UID: \"e80edd8c-d4a2-4bda-9ce6-029183c3ede1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.147005 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.146913 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f3b2bba9-377f-4663-9c77-95f1a1e8910c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.147145 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.147051 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3b2bba9-377f-4663-9c77-95f1a1e8910c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.147145 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.147071 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f3b2bba9-377f-4663-9c77-95f1a1e8910c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.147145 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.147091 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3b2bba9-377f-4663-9c77-95f1a1e8910c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.147145 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.147120 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f3b2bba9-377f-4663-9c77-95f1a1e8910c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.147145 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.147139 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-889fz\" (UniqueName: \"kubernetes.io/projected/f3b2bba9-377f-4663-9c77-95f1a1e8910c-kube-api-access-889fz\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.147394 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.147277 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f3b2bba9-377f-4663-9c77-95f1a1e8910c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.147846 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.147824 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f3b2bba9-377f-4663-9c77-95f1a1e8910c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.149272 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.149250 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3b2bba9-377f-4663-9c77-95f1a1e8910c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.149505 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.149478 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f3b2bba9-377f-4663-9c77-95f1a1e8910c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.149785 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.149766 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f3b2bba9-377f-4663-9c77-95f1a1e8910c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.149833 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.149787 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f3b2bba9-377f-4663-9c77-95f1a1e8910c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.158103 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.158074 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-889fz\" (UniqueName: \"kubernetes.io/projected/f3b2bba9-377f-4663-9c77-95f1a1e8910c-kube-api-access-889fz\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.161735 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.161707 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:11.169513 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.169481 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3b2bba9-377f-4663-9c77-95f1a1e8910c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ncpzc\" (UID: \"f3b2bba9-377f-4663-9c77-95f1a1e8910c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.265129 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.262125 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:11.322234 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.321837 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w"] Jul 12 18:30:11.323128 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:30:11.323098 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode80edd8c_d4a2_4bda_9ce6_029183c3ede1.slice/crio-aa3663c072348ff0f4a38cafa1258518c83e1fd168bc830ad9a4b24e6b3fb505 WatchSource:0}: Error finding container aa3663c072348ff0f4a38cafa1258518c83e1fd168bc830ad9a4b24e6b3fb505: Status 404 returned error can't find the container with id aa3663c072348ff0f4a38cafa1258518c83e1fd168bc830ad9a4b24e6b3fb505 Jul 12 18:30:11.327070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.326473 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 18:30:11.426354 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.426319 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc"] Jul 12 18:30:11.428489 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:30:11.428463 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3b2bba9_377f_4663_9c77_95f1a1e8910c.slice/crio-7c3c8d36cc205c875543c65a88c192b4f9e9ef6e885ba545a40e2504b153a328 WatchSource:0}: Error finding container 7c3c8d36cc205c875543c65a88c192b4f9e9ef6e885ba545a40e2504b153a328: Status 404 returned error can't find the container with id 7c3c8d36cc205c875543c65a88c192b4f9e9ef6e885ba545a40e2504b153a328 Jul 12 18:30:11.764388 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.764294 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" event={"ID":"f3b2bba9-377f-4663-9c77-95f1a1e8910c","Type":"ContainerStarted","Data":"7c3c8d36cc205c875543c65a88c192b4f9e9ef6e885ba545a40e2504b153a328"} Jul 12 18:30:11.765388 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:11.765367 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" event={"ID":"e80edd8c-d4a2-4bda-9ce6-029183c3ede1","Type":"ContainerStarted","Data":"aa3663c072348ff0f4a38cafa1258518c83e1fd168bc830ad9a4b24e6b3fb505"} Jul 12 18:30:14.983806 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:14.983757 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:30:14.984141 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:14.983846 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:30:14.984141 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:14.983877 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:30:14.985587 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:14.985558 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:30:14.985687 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:14.985616 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:30:15.792919 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:15.792866 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" event={"ID":"e80edd8c-d4a2-4bda-9ce6-029183c3ede1","Type":"ContainerStarted","Data":"73cba66a54280927c8abb2d824a9df2f0adb136fc19f377b2d00cc1c82a70a1a"} Jul 12 18:30:15.796459 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:15.796423 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" event={"ID":"f3b2bba9-377f-4663-9c77-95f1a1e8910c","Type":"ContainerStarted","Data":"3eab982b265bab88ef6f5cf464e1c19189bcda4eada2eff7e6f1c8d1fe290645"} Jul 12 18:30:15.796716 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:15.796693 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:15.819713 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:15.819656 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" podStartSLOduration=2.1628582 podStartE2EDuration="5.819640425s" totalImagesPullingTime="3.656782225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:30:15.818010472 +0000 UTC m=+307.017477502" watchObservedRunningTime="2026-07-12 18:30:15.819640425 +0000 UTC m=+307.019107476" Jul 12 18:30:15.841449 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:15.841392 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" podStartSLOduration=2.286503688 podStartE2EDuration="5.841372932s" totalImagesPullingTime="3.554869244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:30:15.839569353 +0000 UTC m=+307.039036408" watchObservedRunningTime="2026-07-12 18:30:15.841372932 +0000 UTC m=+307.040839959" Jul 12 18:30:16.162423 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:16.162344 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:16.167229 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:16.167202 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:16.802596 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:16.802554 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:16.803414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:16.803392 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mdl9w" Jul 12 18:30:16.804373 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:16.804350 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ncpzc" Jul 12 18:30:19.755155 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:19.755125 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-6f7h9" Jul 12 18:30:19.757365 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:30:19.757344 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-l5nxv" Jul 12 18:31:19.240108 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.240027 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8"] Jul 12 18:31:19.242852 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.242832 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" Jul 12 18:31:19.245320 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.245293 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 18:31:19.246229 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.246207 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 18:31:19.246350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.246207 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2dvvh\"" Jul 12 18:31:19.251693 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.251669 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8"] Jul 12 18:31:19.348509 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.348465 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh2fv\" (UniqueName: \"kubernetes.io/projected/21889432-f78d-4a4a-9061-64d3d0958737-kube-api-access-fh2fv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8\" (UID: \"21889432-f78d-4a4a-9061-64d3d0958737\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" Jul 12 18:31:19.348691 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.348520 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21889432-f78d-4a4a-9061-64d3d0958737-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8\" (UID: \"21889432-f78d-4a4a-9061-64d3d0958737\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" Jul 12 18:31:19.348691 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.348672 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21889432-f78d-4a4a-9061-64d3d0958737-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8\" (UID: \"21889432-f78d-4a4a-9061-64d3d0958737\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" Jul 12 18:31:19.449604 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.449559 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh2fv\" (UniqueName: \"kubernetes.io/projected/21889432-f78d-4a4a-9061-64d3d0958737-kube-api-access-fh2fv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8\" (UID: \"21889432-f78d-4a4a-9061-64d3d0958737\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" Jul 12 18:31:19.449820 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.449615 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21889432-f78d-4a4a-9061-64d3d0958737-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8\" (UID: \"21889432-f78d-4a4a-9061-64d3d0958737\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" Jul 12 18:31:19.449820 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.449675 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21889432-f78d-4a4a-9061-64d3d0958737-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8\" (UID: \"21889432-f78d-4a4a-9061-64d3d0958737\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" Jul 12 18:31:19.450095 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.450070 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21889432-f78d-4a4a-9061-64d3d0958737-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8\" (UID: \"21889432-f78d-4a4a-9061-64d3d0958737\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" Jul 12 18:31:19.450167 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.450103 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21889432-f78d-4a4a-9061-64d3d0958737-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8\" (UID: \"21889432-f78d-4a4a-9061-64d3d0958737\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" Jul 12 18:31:19.459252 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.459220 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh2fv\" (UniqueName: \"kubernetes.io/projected/21889432-f78d-4a4a-9061-64d3d0958737-kube-api-access-fh2fv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8\" (UID: \"21889432-f78d-4a4a-9061-64d3d0958737\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" Jul 12 18:31:19.553434 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.553397 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" Jul 12 18:31:19.695303 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.695253 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8"] Jul 12 18:31:19.699136 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:31:19.699106 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21889432_f78d_4a4a_9061_64d3d0958737.slice/crio-24d9a0eb6a1dc71759266b8220ceda5c8aecad320df9b0f871c63c329b78e047 WatchSource:0}: Error finding container 24d9a0eb6a1dc71759266b8220ceda5c8aecad320df9b0f871c63c329b78e047: Status 404 returned error can't find the container with id 24d9a0eb6a1dc71759266b8220ceda5c8aecad320df9b0f871c63c329b78e047 Jul 12 18:31:19.828000 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.827960 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx"] Jul 12 18:31:19.830641 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.830615 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" Jul 12 18:31:19.839511 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.839482 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx"] Jul 12 18:31:19.953767 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.953712 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx\" (UID: \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" Jul 12 18:31:19.953977 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.953788 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87kd9\" (UniqueName: \"kubernetes.io/projected/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-kube-api-access-87kd9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx\" (UID: \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" Jul 12 18:31:19.953977 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:19.953826 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx\" (UID: \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" Jul 12 18:31:20.054843 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.054785 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx\" (UID: \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" Jul 12 18:31:20.054843 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.054857 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87kd9\" (UniqueName: \"kubernetes.io/projected/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-kube-api-access-87kd9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx\" (UID: \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" Jul 12 18:31:20.055093 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.054882 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx\" (UID: \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" Jul 12 18:31:20.055279 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.055258 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx\" (UID: \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" Jul 12 18:31:20.055316 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.055270 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx\" (UID: \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" Jul 12 18:31:20.062876 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.062802 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87kd9\" (UniqueName: \"kubernetes.io/projected/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-kube-api-access-87kd9\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx\" (UID: \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" Jul 12 18:31:20.109425 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.109388 2585 generic.go:356] "Generic (PLEG): container finished" podID="21889432-f78d-4a4a-9061-64d3d0958737" containerID="3dadb7a5cfe161e3b90838b364907c9de4cb0f7d1ed8095692037c840d6adaad" exitCode=0 Jul 12 18:31:20.109579 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.109475 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" event={"ID":"21889432-f78d-4a4a-9061-64d3d0958737","Type":"ContainerDied","Data":"3dadb7a5cfe161e3b90838b364907c9de4cb0f7d1ed8095692037c840d6adaad"} Jul 12 18:31:20.109579 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.109555 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" event={"ID":"21889432-f78d-4a4a-9061-64d3d0958737","Type":"ContainerStarted","Data":"24d9a0eb6a1dc71759266b8220ceda5c8aecad320df9b0f871c63c329b78e047"} Jul 12 18:31:20.171871 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.171831 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" Jul 12 18:31:20.308598 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.308557 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx"] Jul 12 18:31:20.312092 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:31:20.312065 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3e22a16_2734_45fb_9e1f_cfba0060dfa7.slice/crio-0cdf790c83655d9a931851183581e26b00fa73ab1e042d3a43e3a6f8a7c49952 WatchSource:0}: Error finding container 0cdf790c83655d9a931851183581e26b00fa73ab1e042d3a43e3a6f8a7c49952: Status 404 returned error can't find the container with id 0cdf790c83655d9a931851183581e26b00fa73ab1e042d3a43e3a6f8a7c49952 Jul 12 18:31:20.428093 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.428056 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92"] Jul 12 18:31:20.430654 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.430634 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" Jul 12 18:31:20.439311 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.439279 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92"] Jul 12 18:31:20.560263 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.560221 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd891299-c307-491b-b9bf-bd6058d101cc-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92\" (UID: \"fd891299-c307-491b-b9bf-bd6058d101cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" Jul 12 18:31:20.560466 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.560281 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64p4z\" (UniqueName: \"kubernetes.io/projected/fd891299-c307-491b-b9bf-bd6058d101cc-kube-api-access-64p4z\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92\" (UID: \"fd891299-c307-491b-b9bf-bd6058d101cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" Jul 12 18:31:20.560466 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.560406 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd891299-c307-491b-b9bf-bd6058d101cc-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92\" (UID: \"fd891299-c307-491b-b9bf-bd6058d101cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" Jul 12 18:31:20.661243 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.661151 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd891299-c307-491b-b9bf-bd6058d101cc-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92\" (UID: \"fd891299-c307-491b-b9bf-bd6058d101cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" Jul 12 18:31:20.661243 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.661195 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd891299-c307-491b-b9bf-bd6058d101cc-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92\" (UID: \"fd891299-c307-491b-b9bf-bd6058d101cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" Jul 12 18:31:20.661486 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.661331 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64p4z\" (UniqueName: \"kubernetes.io/projected/fd891299-c307-491b-b9bf-bd6058d101cc-kube-api-access-64p4z\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92\" (UID: \"fd891299-c307-491b-b9bf-bd6058d101cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" Jul 12 18:31:20.661617 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.661595 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd891299-c307-491b-b9bf-bd6058d101cc-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92\" (UID: \"fd891299-c307-491b-b9bf-bd6058d101cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" Jul 12 18:31:20.661675 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.661636 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd891299-c307-491b-b9bf-bd6058d101cc-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92\" (UID: \"fd891299-c307-491b-b9bf-bd6058d101cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" Jul 12 18:31:20.669696 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.669661 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64p4z\" (UniqueName: \"kubernetes.io/projected/fd891299-c307-491b-b9bf-bd6058d101cc-kube-api-access-64p4z\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92\" (UID: \"fd891299-c307-491b-b9bf-bd6058d101cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" Jul 12 18:31:20.779689 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.779661 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" Jul 12 18:31:20.826179 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.826145 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f"] Jul 12 18:31:20.828924 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.828903 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" Jul 12 18:31:20.844308 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.843972 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f"] Jul 12 18:31:20.938956 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.938810 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92"] Jul 12 18:31:20.941209 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:31:20.941180 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd891299_c307_491b_b9bf_bd6058d101cc.slice/crio-fec67063af8e096a13722c3abe968722553c0ae48f58f132f53e85bf6c67b487 WatchSource:0}: Error finding container fec67063af8e096a13722c3abe968722553c0ae48f58f132f53e85bf6c67b487: Status 404 returned error can't find the container with id fec67063af8e096a13722c3abe968722553c0ae48f58f132f53e85bf6c67b487 Jul 12 18:31:20.965385 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.965347 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c77cdb1-4b62-47b7-b887-363942e724ff-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f\" (UID: \"0c77cdb1-4b62-47b7-b887-363942e724ff\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" Jul 12 18:31:20.965562 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.965413 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c77cdb1-4b62-47b7-b887-363942e724ff-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f\" (UID: \"0c77cdb1-4b62-47b7-b887-363942e724ff\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" Jul 12 18:31:20.965562 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:20.965493 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t8df\" (UniqueName: \"kubernetes.io/projected/0c77cdb1-4b62-47b7-b887-363942e724ff-kube-api-access-2t8df\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f\" (UID: \"0c77cdb1-4b62-47b7-b887-363942e724ff\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" Jul 12 18:31:21.066598 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.066558 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c77cdb1-4b62-47b7-b887-363942e724ff-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f\" (UID: \"0c77cdb1-4b62-47b7-b887-363942e724ff\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" Jul 12 18:31:21.066773 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.066632 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t8df\" (UniqueName: \"kubernetes.io/projected/0c77cdb1-4b62-47b7-b887-363942e724ff-kube-api-access-2t8df\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f\" (UID: \"0c77cdb1-4b62-47b7-b887-363942e724ff\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" Jul 12 18:31:21.066773 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.066723 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c77cdb1-4b62-47b7-b887-363942e724ff-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f\" (UID: \"0c77cdb1-4b62-47b7-b887-363942e724ff\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" Jul 12 18:31:21.067027 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.067004 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c77cdb1-4b62-47b7-b887-363942e724ff-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f\" (UID: \"0c77cdb1-4b62-47b7-b887-363942e724ff\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" Jul 12 18:31:21.067124 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.067066 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c77cdb1-4b62-47b7-b887-363942e724ff-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f\" (UID: \"0c77cdb1-4b62-47b7-b887-363942e724ff\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" Jul 12 18:31:21.075016 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.074987 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t8df\" (UniqueName: \"kubernetes.io/projected/0c77cdb1-4b62-47b7-b887-363942e724ff-kube-api-access-2t8df\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f\" (UID: \"0c77cdb1-4b62-47b7-b887-363942e724ff\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" Jul 12 18:31:21.117376 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.117337 2585 generic.go:356] "Generic (PLEG): container finished" podID="fd891299-c307-491b-b9bf-bd6058d101cc" containerID="69c2cf9535e77ebb1a595bbacf1f90d88f9b30eb75924daad2abcc83fa7a95ba" exitCode=0 Jul 12 18:31:21.117570 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.117427 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" event={"ID":"fd891299-c307-491b-b9bf-bd6058d101cc","Type":"ContainerDied","Data":"69c2cf9535e77ebb1a595bbacf1f90d88f9b30eb75924daad2abcc83fa7a95ba"} Jul 12 18:31:21.117570 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.117483 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" event={"ID":"fd891299-c307-491b-b9bf-bd6058d101cc","Type":"ContainerStarted","Data":"fec67063af8e096a13722c3abe968722553c0ae48f58f132f53e85bf6c67b487"} Jul 12 18:31:21.119302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.119276 2585 generic.go:356] "Generic (PLEG): container finished" podID="f3e22a16-2734-45fb-9e1f-cfba0060dfa7" containerID="648f7eb93ab5e8005883da54e4430f711f572514c43b9e24c1bd3c8bffe01ee9" exitCode=0 Jul 12 18:31:21.119427 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.119366 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" event={"ID":"f3e22a16-2734-45fb-9e1f-cfba0060dfa7","Type":"ContainerDied","Data":"648f7eb93ab5e8005883da54e4430f711f572514c43b9e24c1bd3c8bffe01ee9"} Jul 12 18:31:21.119427 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.119414 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" event={"ID":"f3e22a16-2734-45fb-9e1f-cfba0060dfa7","Type":"ContainerStarted","Data":"0cdf790c83655d9a931851183581e26b00fa73ab1e042d3a43e3a6f8a7c49952"} Jul 12 18:31:21.121455 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.121417 2585 generic.go:356] "Generic (PLEG): container finished" podID="21889432-f78d-4a4a-9061-64d3d0958737" containerID="ddab171cb04ec7d2a1832a1f09a3ca3c7bba97d7abd7e3018f698a0952353ee4" exitCode=0 Jul 12 18:31:21.121455 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.121455 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" event={"ID":"21889432-f78d-4a4a-9061-64d3d0958737","Type":"ContainerDied","Data":"ddab171cb04ec7d2a1832a1f09a3ca3c7bba97d7abd7e3018f698a0952353ee4"} Jul 12 18:31:21.149315 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.149280 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" Jul 12 18:31:21.296181 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:21.296148 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f"] Jul 12 18:31:21.306878 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:31:21.306841 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c77cdb1_4b62_47b7_b887_363942e724ff.slice/crio-f870e1b746e5fbd4e29bfb30f456eb15bbca89ec91687d774d28eca8e1cc8c6a WatchSource:0}: Error finding container f870e1b746e5fbd4e29bfb30f456eb15bbca89ec91687d774d28eca8e1cc8c6a: Status 404 returned error can't find the container with id f870e1b746e5fbd4e29bfb30f456eb15bbca89ec91687d774d28eca8e1cc8c6a Jul 12 18:31:22.129149 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:22.129111 2585 generic.go:356] "Generic (PLEG): container finished" podID="f3e22a16-2734-45fb-9e1f-cfba0060dfa7" containerID="fc75121737cc9f83e5241b960e4cdade3b5e179587702def04929972c543493f" exitCode=0 Jul 12 18:31:22.129619 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:22.129202 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" event={"ID":"f3e22a16-2734-45fb-9e1f-cfba0060dfa7","Type":"ContainerDied","Data":"fc75121737cc9f83e5241b960e4cdade3b5e179587702def04929972c543493f"} Jul 12 18:31:22.130809 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:22.130761 2585 generic.go:356] "Generic (PLEG): container finished" podID="0c77cdb1-4b62-47b7-b887-363942e724ff" containerID="1741d0630019af080307fc1fbbd9e26db65dd034a52215b0ec553304f495e07b" exitCode=0 Jul 12 18:31:22.130877 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:22.130850 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" event={"ID":"0c77cdb1-4b62-47b7-b887-363942e724ff","Type":"ContainerDied","Data":"1741d0630019af080307fc1fbbd9e26db65dd034a52215b0ec553304f495e07b"} Jul 12 18:31:22.130922 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:22.130908 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" event={"ID":"0c77cdb1-4b62-47b7-b887-363942e724ff","Type":"ContainerStarted","Data":"f870e1b746e5fbd4e29bfb30f456eb15bbca89ec91687d774d28eca8e1cc8c6a"} Jul 12 18:31:22.133334 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:22.133215 2585 generic.go:356] "Generic (PLEG): container finished" podID="21889432-f78d-4a4a-9061-64d3d0958737" containerID="381715546c52d3c177ba5e88e871b1100cfd702d19ebf6745550ef520f8972bb" exitCode=0 Jul 12 18:31:22.133334 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:22.133302 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" event={"ID":"21889432-f78d-4a4a-9061-64d3d0958737","Type":"ContainerDied","Data":"381715546c52d3c177ba5e88e871b1100cfd702d19ebf6745550ef520f8972bb"} Jul 12 18:31:22.135291 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:22.135268 2585 generic.go:356] "Generic (PLEG): container finished" podID="fd891299-c307-491b-b9bf-bd6058d101cc" containerID="ee528a3a40abd08b72e032b05700070df71e44072c0176b3c9b9570acaff9eb3" exitCode=0 Jul 12 18:31:22.135382 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:22.135345 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" event={"ID":"fd891299-c307-491b-b9bf-bd6058d101cc","Type":"ContainerDied","Data":"ee528a3a40abd08b72e032b05700070df71e44072c0176b3c9b9570acaff9eb3"} Jul 12 18:31:23.144306 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.144273 2585 generic.go:356] "Generic (PLEG): container finished" podID="f3e22a16-2734-45fb-9e1f-cfba0060dfa7" containerID="040db49883ce770f63ebcc875fe0602a006cba889568d02f6231cb538cadb3ad" exitCode=0 Jul 12 18:31:23.144712 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.144353 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" event={"ID":"f3e22a16-2734-45fb-9e1f-cfba0060dfa7","Type":"ContainerDied","Data":"040db49883ce770f63ebcc875fe0602a006cba889568d02f6231cb538cadb3ad"} Jul 12 18:31:23.146288 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.146259 2585 generic.go:356] "Generic (PLEG): container finished" podID="0c77cdb1-4b62-47b7-b887-363942e724ff" containerID="4ab2aa83b985fe700be81ff7c6063b918c4424d1d3cefbcd5d344ca6b3979fb9" exitCode=0 Jul 12 18:31:23.146418 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.146338 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" event={"ID":"0c77cdb1-4b62-47b7-b887-363942e724ff","Type":"ContainerDied","Data":"4ab2aa83b985fe700be81ff7c6063b918c4424d1d3cefbcd5d344ca6b3979fb9"} Jul 12 18:31:23.152279 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.152186 2585 generic.go:356] "Generic (PLEG): container finished" podID="fd891299-c307-491b-b9bf-bd6058d101cc" containerID="eef8145c61d022f7a7119e03a3b7890da83a9f30dfcf93d8820db9b2a0f87ded" exitCode=0 Jul 12 18:31:23.152279 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.152250 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" event={"ID":"fd891299-c307-491b-b9bf-bd6058d101cc","Type":"ContainerDied","Data":"eef8145c61d022f7a7119e03a3b7890da83a9f30dfcf93d8820db9b2a0f87ded"} Jul 12 18:31:23.287796 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.287762 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" Jul 12 18:31:23.387245 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.387212 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21889432-f78d-4a4a-9061-64d3d0958737-util\") pod \"21889432-f78d-4a4a-9061-64d3d0958737\" (UID: \"21889432-f78d-4a4a-9061-64d3d0958737\") " Jul 12 18:31:23.387442 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.387256 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fh2fv\" (UniqueName: \"kubernetes.io/projected/21889432-f78d-4a4a-9061-64d3d0958737-kube-api-access-fh2fv\") pod \"21889432-f78d-4a4a-9061-64d3d0958737\" (UID: \"21889432-f78d-4a4a-9061-64d3d0958737\") " Jul 12 18:31:23.387442 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.387368 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21889432-f78d-4a4a-9061-64d3d0958737-bundle\") pod \"21889432-f78d-4a4a-9061-64d3d0958737\" (UID: \"21889432-f78d-4a4a-9061-64d3d0958737\") " Jul 12 18:31:23.387858 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.387831 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21889432-f78d-4a4a-9061-64d3d0958737-bundle" (OuterVolumeSpecName: "bundle") pod "21889432-f78d-4a4a-9061-64d3d0958737" (UID: "21889432-f78d-4a4a-9061-64d3d0958737"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:31:23.389596 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.389570 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21889432-f78d-4a4a-9061-64d3d0958737-kube-api-access-fh2fv" (OuterVolumeSpecName: "kube-api-access-fh2fv") pod "21889432-f78d-4a4a-9061-64d3d0958737" (UID: "21889432-f78d-4a4a-9061-64d3d0958737"). InnerVolumeSpecName "kube-api-access-fh2fv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:31:23.392875 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.392844 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21889432-f78d-4a4a-9061-64d3d0958737-util" (OuterVolumeSpecName: "util") pod "21889432-f78d-4a4a-9061-64d3d0958737" (UID: "21889432-f78d-4a4a-9061-64d3d0958737"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:31:23.488235 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.488184 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21889432-f78d-4a4a-9061-64d3d0958737-util\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:31:23.488235 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.488229 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fh2fv\" (UniqueName: \"kubernetes.io/projected/21889432-f78d-4a4a-9061-64d3d0958737-kube-api-access-fh2fv\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:31:23.488235 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:23.488241 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21889432-f78d-4a4a-9061-64d3d0958737-bundle\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:31:24.160505 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.160468 2585 generic.go:356] "Generic (PLEG): container finished" podID="0c77cdb1-4b62-47b7-b887-363942e724ff" containerID="0e1e722c07e6cc2186a7ef058a116eee8208f65cce02c3cba9499a3bf637731e" exitCode=0 Jul 12 18:31:24.160959 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.160556 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" event={"ID":"0c77cdb1-4b62-47b7-b887-363942e724ff","Type":"ContainerDied","Data":"0e1e722c07e6cc2186a7ef058a116eee8208f65cce02c3cba9499a3bf637731e"} Jul 12 18:31:24.162612 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.162588 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" Jul 12 18:31:24.162729 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.162604 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8" event={"ID":"21889432-f78d-4a4a-9061-64d3d0958737","Type":"ContainerDied","Data":"24d9a0eb6a1dc71759266b8220ceda5c8aecad320df9b0f871c63c329b78e047"} Jul 12 18:31:24.162729 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.162653 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24d9a0eb6a1dc71759266b8220ceda5c8aecad320df9b0f871c63c329b78e047" Jul 12 18:31:24.333533 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.333498 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" Jul 12 18:31:24.336429 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.336399 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" Jul 12 18:31:24.396442 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.396395 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87kd9\" (UniqueName: \"kubernetes.io/projected/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-kube-api-access-87kd9\") pod \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\" (UID: \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\") " Jul 12 18:31:24.396619 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.396478 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-util\") pod \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\" (UID: \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\") " Jul 12 18:31:24.396619 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.396496 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64p4z\" (UniqueName: \"kubernetes.io/projected/fd891299-c307-491b-b9bf-bd6058d101cc-kube-api-access-64p4z\") pod \"fd891299-c307-491b-b9bf-bd6058d101cc\" (UID: \"fd891299-c307-491b-b9bf-bd6058d101cc\") " Jul 12 18:31:24.396693 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.396625 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd891299-c307-491b-b9bf-bd6058d101cc-util\") pod \"fd891299-c307-491b-b9bf-bd6058d101cc\" (UID: \"fd891299-c307-491b-b9bf-bd6058d101cc\") " Jul 12 18:31:24.396693 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.396677 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd891299-c307-491b-b9bf-bd6058d101cc-bundle\") pod \"fd891299-c307-491b-b9bf-bd6058d101cc\" (UID: \"fd891299-c307-491b-b9bf-bd6058d101cc\") " Jul 12 18:31:24.396765 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.396719 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-bundle\") pod \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\" (UID: \"f3e22a16-2734-45fb-9e1f-cfba0060dfa7\") " Jul 12 18:31:24.397287 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.397251 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-bundle" (OuterVolumeSpecName: "bundle") pod "f3e22a16-2734-45fb-9e1f-cfba0060dfa7" (UID: "f3e22a16-2734-45fb-9e1f-cfba0060dfa7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:31:24.397421 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.397392 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd891299-c307-491b-b9bf-bd6058d101cc-bundle" (OuterVolumeSpecName: "bundle") pod "fd891299-c307-491b-b9bf-bd6058d101cc" (UID: "fd891299-c307-491b-b9bf-bd6058d101cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:31:24.398970 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.398893 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-kube-api-access-87kd9" (OuterVolumeSpecName: "kube-api-access-87kd9") pod "f3e22a16-2734-45fb-9e1f-cfba0060dfa7" (UID: "f3e22a16-2734-45fb-9e1f-cfba0060dfa7"). InnerVolumeSpecName "kube-api-access-87kd9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:31:24.399095 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.399076 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd891299-c307-491b-b9bf-bd6058d101cc-kube-api-access-64p4z" (OuterVolumeSpecName: "kube-api-access-64p4z") pod "fd891299-c307-491b-b9bf-bd6058d101cc" (UID: "fd891299-c307-491b-b9bf-bd6058d101cc"). InnerVolumeSpecName "kube-api-access-64p4z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:31:24.402557 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.402528 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-util" (OuterVolumeSpecName: "util") pod "f3e22a16-2734-45fb-9e1f-cfba0060dfa7" (UID: "f3e22a16-2734-45fb-9e1f-cfba0060dfa7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:31:24.403452 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.403428 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd891299-c307-491b-b9bf-bd6058d101cc-util" (OuterVolumeSpecName: "util") pod "fd891299-c307-491b-b9bf-bd6058d101cc" (UID: "fd891299-c307-491b-b9bf-bd6058d101cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:31:24.498320 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.498228 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87kd9\" (UniqueName: \"kubernetes.io/projected/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-kube-api-access-87kd9\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:31:24.498320 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.498263 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-util\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:31:24.498320 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.498274 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64p4z\" (UniqueName: \"kubernetes.io/projected/fd891299-c307-491b-b9bf-bd6058d101cc-kube-api-access-64p4z\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:31:24.498320 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.498283 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fd891299-c307-491b-b9bf-bd6058d101cc-util\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:31:24.498320 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.498291 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fd891299-c307-491b-b9bf-bd6058d101cc-bundle\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:31:24.498320 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:24.498299 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3e22a16-2734-45fb-9e1f-cfba0060dfa7-bundle\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:31:25.169563 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.169530 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" Jul 12 18:31:25.169563 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.169543 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx" event={"ID":"f3e22a16-2734-45fb-9e1f-cfba0060dfa7","Type":"ContainerDied","Data":"0cdf790c83655d9a931851183581e26b00fa73ab1e042d3a43e3a6f8a7c49952"} Jul 12 18:31:25.170110 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.169592 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cdf790c83655d9a931851183581e26b00fa73ab1e042d3a43e3a6f8a7c49952" Jul 12 18:31:25.171644 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.171621 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" Jul 12 18:31:25.171782 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.171637 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92" event={"ID":"fd891299-c307-491b-b9bf-bd6058d101cc","Type":"ContainerDied","Data":"fec67063af8e096a13722c3abe968722553c0ae48f58f132f53e85bf6c67b487"} Jul 12 18:31:25.171782 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.171695 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fec67063af8e096a13722c3abe968722553c0ae48f58f132f53e85bf6c67b487" Jul 12 18:31:25.314107 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.314078 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" Jul 12 18:31:25.405179 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.405146 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c77cdb1-4b62-47b7-b887-363942e724ff-bundle\") pod \"0c77cdb1-4b62-47b7-b887-363942e724ff\" (UID: \"0c77cdb1-4b62-47b7-b887-363942e724ff\") " Jul 12 18:31:25.405372 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.405188 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c77cdb1-4b62-47b7-b887-363942e724ff-util\") pod \"0c77cdb1-4b62-47b7-b887-363942e724ff\" (UID: \"0c77cdb1-4b62-47b7-b887-363942e724ff\") " Jul 12 18:31:25.405372 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.405273 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2t8df\" (UniqueName: \"kubernetes.io/projected/0c77cdb1-4b62-47b7-b887-363942e724ff-kube-api-access-2t8df\") pod \"0c77cdb1-4b62-47b7-b887-363942e724ff\" (UID: \"0c77cdb1-4b62-47b7-b887-363942e724ff\") " Jul 12 18:31:25.405753 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.405714 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c77cdb1-4b62-47b7-b887-363942e724ff-bundle" (OuterVolumeSpecName: "bundle") pod "0c77cdb1-4b62-47b7-b887-363942e724ff" (UID: "0c77cdb1-4b62-47b7-b887-363942e724ff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:31:25.407489 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.407461 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c77cdb1-4b62-47b7-b887-363942e724ff-kube-api-access-2t8df" (OuterVolumeSpecName: "kube-api-access-2t8df") pod "0c77cdb1-4b62-47b7-b887-363942e724ff" (UID: "0c77cdb1-4b62-47b7-b887-363942e724ff"). InnerVolumeSpecName "kube-api-access-2t8df". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:31:25.412996 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.412957 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c77cdb1-4b62-47b7-b887-363942e724ff-util" (OuterVolumeSpecName: "util") pod "0c77cdb1-4b62-47b7-b887-363942e724ff" (UID: "0c77cdb1-4b62-47b7-b887-363942e724ff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:31:25.506399 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.506296 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t8df\" (UniqueName: \"kubernetes.io/projected/0c77cdb1-4b62-47b7-b887-363942e724ff-kube-api-access-2t8df\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:31:25.506399 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.506335 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c77cdb1-4b62-47b7-b887-363942e724ff-bundle\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:31:25.506399 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:25.506346 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c77cdb1-4b62-47b7-b887-363942e724ff-util\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:31:26.178450 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:26.178410 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" event={"ID":"0c77cdb1-4b62-47b7-b887-363942e724ff","Type":"ContainerDied","Data":"f870e1b746e5fbd4e29bfb30f456eb15bbca89ec91687d774d28eca8e1cc8c6a"} Jul 12 18:31:26.178450 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:26.178441 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f" Jul 12 18:31:26.178901 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:26.178490 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f870e1b746e5fbd4e29bfb30f456eb15bbca89ec91687d774d28eca8e1cc8c6a" Jul 12 18:31:44.226622 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.226580 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kp6lf"] Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.226902 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3e22a16-2734-45fb-9e1f-cfba0060dfa7" containerName="util" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.226913 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3e22a16-2734-45fb-9e1f-cfba0060dfa7" containerName="util" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.226920 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c77cdb1-4b62-47b7-b887-363942e724ff" containerName="pull" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.226925 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c77cdb1-4b62-47b7-b887-363942e724ff" containerName="pull" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.226951 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21889432-f78d-4a4a-9061-64d3d0958737" containerName="util" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.226960 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="21889432-f78d-4a4a-9061-64d3d0958737" containerName="util" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.226970 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3e22a16-2734-45fb-9e1f-cfba0060dfa7" containerName="pull" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.226975 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3e22a16-2734-45fb-9e1f-cfba0060dfa7" containerName="pull" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.226982 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21889432-f78d-4a4a-9061-64d3d0958737" containerName="extract" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.226988 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="21889432-f78d-4a4a-9061-64d3d0958737" containerName="extract" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.226995 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21889432-f78d-4a4a-9061-64d3d0958737" containerName="pull" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227000 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="21889432-f78d-4a4a-9061-64d3d0958737" containerName="pull" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227005 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd891299-c307-491b-b9bf-bd6058d101cc" containerName="util" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227010 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd891299-c307-491b-b9bf-bd6058d101cc" containerName="util" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227017 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd891299-c307-491b-b9bf-bd6058d101cc" containerName="pull" Jul 12 18:31:44.227015 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227023 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd891299-c307-491b-b9bf-bd6058d101cc" containerName="pull" Jul 12 18:31:44.227573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227029 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3e22a16-2734-45fb-9e1f-cfba0060dfa7" containerName="extract" Jul 12 18:31:44.227573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227034 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3e22a16-2734-45fb-9e1f-cfba0060dfa7" containerName="extract" Jul 12 18:31:44.227573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227047 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c77cdb1-4b62-47b7-b887-363942e724ff" containerName="extract" Jul 12 18:31:44.227573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227052 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c77cdb1-4b62-47b7-b887-363942e724ff" containerName="extract" Jul 12 18:31:44.227573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227061 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c77cdb1-4b62-47b7-b887-363942e724ff" containerName="util" Jul 12 18:31:44.227573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227066 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c77cdb1-4b62-47b7-b887-363942e724ff" containerName="util" Jul 12 18:31:44.227573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227073 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd891299-c307-491b-b9bf-bd6058d101cc" containerName="extract" Jul 12 18:31:44.227573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227078 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd891299-c307-491b-b9bf-bd6058d101cc" containerName="extract" Jul 12 18:31:44.227573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227128 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3e22a16-2734-45fb-9e1f-cfba0060dfa7" containerName="extract" Jul 12 18:31:44.227573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227136 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd891299-c307-491b-b9bf-bd6058d101cc" containerName="extract" Jul 12 18:31:44.227573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227142 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c77cdb1-4b62-47b7-b887-363942e724ff" containerName="extract" Jul 12 18:31:44.227573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.227147 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="21889432-f78d-4a4a-9061-64d3d0958737" containerName="extract" Jul 12 18:31:44.230266 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.230246 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kp6lf" Jul 12 18:31:44.232563 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.232538 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 18:31:44.232665 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.232641 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 18:31:44.232761 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.232745 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 12 18:31:44.233472 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.233457 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-gpjfp\"" Jul 12 18:31:44.241366 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.241334 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kp6lf"] Jul 12 18:31:44.366501 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.366464 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnrlq\" (UniqueName: \"kubernetes.io/projected/9f2d972d-3329-4b94-b5e8-ede7b89f0649-kube-api-access-hnrlq\") pod \"dns-operator-controller-manager-864dd78c4c-kp6lf\" (UID: \"9f2d972d-3329-4b94-b5e8-ede7b89f0649\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kp6lf" Jul 12 18:31:44.466987 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.466912 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnrlq\" (UniqueName: \"kubernetes.io/projected/9f2d972d-3329-4b94-b5e8-ede7b89f0649-kube-api-access-hnrlq\") pod \"dns-operator-controller-manager-864dd78c4c-kp6lf\" (UID: \"9f2d972d-3329-4b94-b5e8-ede7b89f0649\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kp6lf" Jul 12 18:31:44.477461 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.477377 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnrlq\" (UniqueName: \"kubernetes.io/projected/9f2d972d-3329-4b94-b5e8-ede7b89f0649-kube-api-access-hnrlq\") pod \"dns-operator-controller-manager-864dd78c4c-kp6lf\" (UID: \"9f2d972d-3329-4b94-b5e8-ede7b89f0649\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kp6lf" Jul 12 18:31:44.541381 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.541343 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kp6lf" Jul 12 18:31:44.694196 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:44.694170 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kp6lf"] Jul 12 18:31:44.696078 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:31:44.696042 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f2d972d_3329_4b94_b5e8_ede7b89f0649.slice/crio-9fc5dce6ce996061f17251e785279ac45caaf3a17db651ce235c5de38517789e WatchSource:0}: Error finding container 9fc5dce6ce996061f17251e785279ac45caaf3a17db651ce235c5de38517789e: Status 404 returned error can't find the container with id 9fc5dce6ce996061f17251e785279ac45caaf3a17db651ce235c5de38517789e Jul 12 18:31:45.277317 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:45.277281 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kp6lf" event={"ID":"9f2d972d-3329-4b94-b5e8-ede7b89f0649","Type":"ContainerStarted","Data":"9fc5dce6ce996061f17251e785279ac45caaf3a17db651ce235c5de38517789e"} Jul 12 18:31:46.913148 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:46.913115 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9"] Jul 12 18:31:46.917834 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:46.917808 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" Jul 12 18:31:46.920219 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:46.920195 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-bksdv\"" Jul 12 18:31:46.926351 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:46.926310 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9"] Jul 12 18:31:47.094166 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:47.094118 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxrdg\" (UniqueName: \"kubernetes.io/projected/fc67d088-2fc9-4b39-b500-25d552959873-kube-api-access-rxrdg\") pod \"limitador-operator-controller-manager-859d7b78bd-xsrl9\" (UID: \"fc67d088-2fc9-4b39-b500-25d552959873\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" Jul 12 18:31:47.194724 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:47.194625 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxrdg\" (UniqueName: \"kubernetes.io/projected/fc67d088-2fc9-4b39-b500-25d552959873-kube-api-access-rxrdg\") pod \"limitador-operator-controller-manager-859d7b78bd-xsrl9\" (UID: \"fc67d088-2fc9-4b39-b500-25d552959873\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" Jul 12 18:31:47.204089 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:47.204053 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxrdg\" (UniqueName: \"kubernetes.io/projected/fc67d088-2fc9-4b39-b500-25d552959873-kube-api-access-rxrdg\") pod \"limitador-operator-controller-manager-859d7b78bd-xsrl9\" (UID: \"fc67d088-2fc9-4b39-b500-25d552959873\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" Jul 12 18:31:47.230817 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:47.230774 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" Jul 12 18:31:47.290901 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:47.290861 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kp6lf" event={"ID":"9f2d972d-3329-4b94-b5e8-ede7b89f0649","Type":"ContainerStarted","Data":"d19e7d211d41415250332075155fa0a50885a66f7ea6bf2877330fed3d13c112"} Jul 12 18:31:47.291077 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:47.290968 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kp6lf" Jul 12 18:31:47.309303 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:47.309249 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kp6lf" podStartSLOduration=1.145039254 podStartE2EDuration="3.309231454s" totalImagesPullingTime="2.1641922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:31:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:31:47.307011994 +0000 UTC m=+398.506530739" watchObservedRunningTime="2026-07-12 18:31:47.309231454 +0000 UTC m=+398.508698484" Jul 12 18:31:47.381228 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:47.381199 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9"] Jul 12 18:31:47.383399 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:31:47.383364 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc67d088_2fc9_4b39_b500_25d552959873.slice/crio-8066f750047c051420c94cfbe419f7810786ef7eaf6ba923c189c0a1902365b4 WatchSource:0}: Error finding container 8066f750047c051420c94cfbe419f7810786ef7eaf6ba923c189c0a1902365b4: Status 404 returned error can't find the container with id 8066f750047c051420c94cfbe419f7810786ef7eaf6ba923c189c0a1902365b4 Jul 12 18:31:48.297623 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:48.297590 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" event={"ID":"fc67d088-2fc9-4b39-b500-25d552959873","Type":"ContainerStarted","Data":"8066f750047c051420c94cfbe419f7810786ef7eaf6ba923c189c0a1902365b4"} Jul 12 18:31:50.320007 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:50.319958 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" event={"ID":"fc67d088-2fc9-4b39-b500-25d552959873","Type":"ContainerStarted","Data":"9a9a0f26a0399b29b16c2cbc98cb625604f51c69b0aa59a9c265d6749eee2a67"} Jul 12 18:31:50.320464 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:50.320167 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" Jul 12 18:31:50.336665 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:50.336610 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" podStartSLOduration=2.229030927 podStartE2EDuration="4.336594545s" totalImagesPullingTime="2.107563618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:31:50.336125512 +0000 UTC m=+401.535592542" watchObservedRunningTime="2026-07-12 18:31:50.336594545 +0000 UTC m=+401.536061574" Jul 12 18:31:58.300424 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:31:58.300394 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kp6lf" Jul 12 18:32:00.058454 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.058417 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6"] Jul 12 18:32:00.063275 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.063252 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" Jul 12 18:32:00.068654 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.068627 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gvhmp\"" Jul 12 18:32:00.075709 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.075677 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6"] Jul 12 18:32:00.111042 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.110999 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/713ffc38-dd0f-4f7b-9dbb-5bd17b74295e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-bxwq6\" (UID: \"713ffc38-dd0f-4f7b-9dbb-5bd17b74295e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" Jul 12 18:32:00.111220 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.111095 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s7gf\" (UniqueName: \"kubernetes.io/projected/713ffc38-dd0f-4f7b-9dbb-5bd17b74295e-kube-api-access-9s7gf\") pod \"kuadrant-operator-controller-manager-55467d9f7-bxwq6\" (UID: \"713ffc38-dd0f-4f7b-9dbb-5bd17b74295e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" Jul 12 18:32:00.163992 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.163954 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6"] Jul 12 18:32:00.164246 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:32:00.164226 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-9s7gf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" podUID="713ffc38-dd0f-4f7b-9dbb-5bd17b74295e" Jul 12 18:32:00.212397 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.212351 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/713ffc38-dd0f-4f7b-9dbb-5bd17b74295e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-bxwq6\" (UID: \"713ffc38-dd0f-4f7b-9dbb-5bd17b74295e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" Jul 12 18:32:00.212397 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.212400 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s7gf\" (UniqueName: \"kubernetes.io/projected/713ffc38-dd0f-4f7b-9dbb-5bd17b74295e-kube-api-access-9s7gf\") pod \"kuadrant-operator-controller-manager-55467d9f7-bxwq6\" (UID: \"713ffc38-dd0f-4f7b-9dbb-5bd17b74295e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" Jul 12 18:32:00.212779 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.212756 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/713ffc38-dd0f-4f7b-9dbb-5bd17b74295e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-bxwq6\" (UID: \"713ffc38-dd0f-4f7b-9dbb-5bd17b74295e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" Jul 12 18:32:00.221295 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.221263 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s7gf\" (UniqueName: \"kubernetes.io/projected/713ffc38-dd0f-4f7b-9dbb-5bd17b74295e-kube-api-access-9s7gf\") pod \"kuadrant-operator-controller-manager-55467d9f7-bxwq6\" (UID: \"713ffc38-dd0f-4f7b-9dbb-5bd17b74295e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" Jul 12 18:32:00.240168 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.240126 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6"] Jul 12 18:32:00.247991 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.247955 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6"] Jul 12 18:32:00.289239 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.289202 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj"] Jul 12 18:32:00.292974 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.292948 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9"] Jul 12 18:32:00.293122 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.293075 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" Jul 12 18:32:00.293338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.293303 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" podUID="fc67d088-2fc9-4b39-b500-25d552959873" containerName="manager" containerID="cri-o://9a9a0f26a0399b29b16c2cbc98cb625604f51c69b0aa59a9c265d6749eee2a67" gracePeriod=2 Jul 12 18:32:00.295287 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.295263 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" Jul 12 18:32:00.301576 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.301548 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9"] Jul 12 18:32:00.305771 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.305728 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj"] Jul 12 18:32:00.321923 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.321885 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbvzd"] Jul 12 18:32:00.322441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.322417 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc67d088-2fc9-4b39-b500-25d552959873" containerName="manager" Jul 12 18:32:00.322441 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.322443 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc67d088-2fc9-4b39-b500-25d552959873" containerName="manager" Jul 12 18:32:00.322591 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.322548 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc67d088-2fc9-4b39-b500-25d552959873" containerName="manager" Jul 12 18:32:00.325629 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.325605 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbvzd" Jul 12 18:32:00.343289 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.343256 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbvzd"] Jul 12 18:32:00.355835 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:32:00.355796 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xsrl9\" is forbidden: User \"system:node:ip-10-0-140-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-170.ec2.internal' and this object" podUID="fc67d088-2fc9-4b39-b500-25d552959873" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" Jul 12 18:32:00.373634 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.373605 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" Jul 12 18:32:00.376124 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:32:00.376088 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xsrl9\" is forbidden: User \"system:node:ip-10-0-140-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-170.ec2.internal' and this object" podUID="fc67d088-2fc9-4b39-b500-25d552959873" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" Jul 12 18:32:00.378083 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:32:00.378037 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-bxwq6\" is forbidden: User \"system:node:ip-10-0-140-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-170.ec2.internal' and this object" podUID="713ffc38-dd0f-4f7b-9dbb-5bd17b74295e" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" Jul 12 18:32:00.380317 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.380297 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" Jul 12 18:32:00.383040 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:32:00.382997 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xsrl9\" is forbidden: User \"system:node:ip-10-0-140-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-170.ec2.internal' and this object" podUID="fc67d088-2fc9-4b39-b500-25d552959873" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" Jul 12 18:32:00.385246 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:32:00.385217 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-bxwq6\" is forbidden: User \"system:node:ip-10-0-140-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-170.ec2.internal' and this object" podUID="713ffc38-dd0f-4f7b-9dbb-5bd17b74295e" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" Jul 12 18:32:00.413878 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.413849 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/713ffc38-dd0f-4f7b-9dbb-5bd17b74295e-extensions-socket-volume\") pod \"713ffc38-dd0f-4f7b-9dbb-5bd17b74295e\" (UID: \"713ffc38-dd0f-4f7b-9dbb-5bd17b74295e\") " Jul 12 18:32:00.414020 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.413980 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9s7gf\" (UniqueName: \"kubernetes.io/projected/713ffc38-dd0f-4f7b-9dbb-5bd17b74295e-kube-api-access-9s7gf\") pod \"713ffc38-dd0f-4f7b-9dbb-5bd17b74295e\" (UID: \"713ffc38-dd0f-4f7b-9dbb-5bd17b74295e\") " Jul 12 18:32:00.414166 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.414143 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4wvt\" (UniqueName: \"kubernetes.io/projected/bfb07e88-9932-4ef7-9488-57b79c51578f-kube-api-access-t4wvt\") pod \"limitador-operator-controller-manager-859d7b78bd-kbvzd\" (UID: \"bfb07e88-9932-4ef7-9488-57b79c51578f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbvzd" Jul 12 18:32:00.414261 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.414203 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/713ffc38-dd0f-4f7b-9dbb-5bd17b74295e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "713ffc38-dd0f-4f7b-9dbb-5bd17b74295e" (UID: "713ffc38-dd0f-4f7b-9dbb-5bd17b74295e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:32:00.414261 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.414218 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llhhx\" (UniqueName: \"kubernetes.io/projected/7dcb9a46-f43f-4743-96b1-e6fe401fb31d-kube-api-access-llhhx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gxhnj\" (UID: \"7dcb9a46-f43f-4743-96b1-e6fe401fb31d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" Jul 12 18:32:00.414364 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.414277 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7dcb9a46-f43f-4743-96b1-e6fe401fb31d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gxhnj\" (UID: \"7dcb9a46-f43f-4743-96b1-e6fe401fb31d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" Jul 12 18:32:00.414364 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.414348 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/713ffc38-dd0f-4f7b-9dbb-5bd17b74295e-extensions-socket-volume\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:32:00.416296 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.416268 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/713ffc38-dd0f-4f7b-9dbb-5bd17b74295e-kube-api-access-9s7gf" (OuterVolumeSpecName: "kube-api-access-9s7gf") pod "713ffc38-dd0f-4f7b-9dbb-5bd17b74295e" (UID: "713ffc38-dd0f-4f7b-9dbb-5bd17b74295e"). InnerVolumeSpecName "kube-api-access-9s7gf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:32:00.515520 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.515484 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4wvt\" (UniqueName: \"kubernetes.io/projected/bfb07e88-9932-4ef7-9488-57b79c51578f-kube-api-access-t4wvt\") pod \"limitador-operator-controller-manager-859d7b78bd-kbvzd\" (UID: \"bfb07e88-9932-4ef7-9488-57b79c51578f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbvzd" Jul 12 18:32:00.515721 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.515542 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llhhx\" (UniqueName: \"kubernetes.io/projected/7dcb9a46-f43f-4743-96b1-e6fe401fb31d-kube-api-access-llhhx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gxhnj\" (UID: \"7dcb9a46-f43f-4743-96b1-e6fe401fb31d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" Jul 12 18:32:00.515721 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.515671 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7dcb9a46-f43f-4743-96b1-e6fe401fb31d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gxhnj\" (UID: \"7dcb9a46-f43f-4743-96b1-e6fe401fb31d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" Jul 12 18:32:00.515836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.515819 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9s7gf\" (UniqueName: \"kubernetes.io/projected/713ffc38-dd0f-4f7b-9dbb-5bd17b74295e-kube-api-access-9s7gf\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:32:00.516164 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.516141 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7dcb9a46-f43f-4743-96b1-e6fe401fb31d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gxhnj\" (UID: \"7dcb9a46-f43f-4743-96b1-e6fe401fb31d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" Jul 12 18:32:00.524018 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.523991 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4wvt\" (UniqueName: \"kubernetes.io/projected/bfb07e88-9932-4ef7-9488-57b79c51578f-kube-api-access-t4wvt\") pod \"limitador-operator-controller-manager-859d7b78bd-kbvzd\" (UID: \"bfb07e88-9932-4ef7-9488-57b79c51578f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbvzd" Jul 12 18:32:00.524132 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.523994 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llhhx\" (UniqueName: \"kubernetes.io/projected/7dcb9a46-f43f-4743-96b1-e6fe401fb31d-kube-api-access-llhhx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gxhnj\" (UID: \"7dcb9a46-f43f-4743-96b1-e6fe401fb31d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" Jul 12 18:32:00.547169 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.547140 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" Jul 12 18:32:00.549113 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:32:00.549079 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xsrl9\" is forbidden: User \"system:node:ip-10-0-140-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-170.ec2.internal' and this object" podUID="fc67d088-2fc9-4b39-b500-25d552959873" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" Jul 12 18:32:00.550979 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:32:00.550923 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-bxwq6\" is forbidden: User \"system:node:ip-10-0-140-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-170.ec2.internal' and this object" podUID="713ffc38-dd0f-4f7b-9dbb-5bd17b74295e" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" Jul 12 18:32:00.616668 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.616582 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxrdg\" (UniqueName: \"kubernetes.io/projected/fc67d088-2fc9-4b39-b500-25d552959873-kube-api-access-rxrdg\") pod \"fc67d088-2fc9-4b39-b500-25d552959873\" (UID: \"fc67d088-2fc9-4b39-b500-25d552959873\") " Jul 12 18:32:00.618657 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.618630 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc67d088-2fc9-4b39-b500-25d552959873-kube-api-access-rxrdg" (OuterVolumeSpecName: "kube-api-access-rxrdg") pod "fc67d088-2fc9-4b39-b500-25d552959873" (UID: "fc67d088-2fc9-4b39-b500-25d552959873"). InnerVolumeSpecName "kube-api-access-rxrdg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:32:00.666843 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.666794 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" Jul 12 18:32:00.672906 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.672881 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbvzd" Jul 12 18:32:00.718390 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.718196 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxrdg\" (UniqueName: \"kubernetes.io/projected/fc67d088-2fc9-4b39-b500-25d552959873-kube-api-access-rxrdg\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:32:00.822023 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.821945 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj"] Jul 12 18:32:00.826120 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:32:00.826088 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dcb9a46_f43f_4743_96b1_e6fe401fb31d.slice/crio-d1de023189fcf961a3f38de43ff910e3c301299c26420845b9e16200d7083ef6 WatchSource:0}: Error finding container d1de023189fcf961a3f38de43ff910e3c301299c26420845b9e16200d7083ef6: Status 404 returned error can't find the container with id d1de023189fcf961a3f38de43ff910e3c301299c26420845b9e16200d7083ef6 Jul 12 18:32:00.848646 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:00.848619 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbvzd"] Jul 12 18:32:00.850539 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:32:00.850501 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb07e88_9932_4ef7_9488_57b79c51578f.slice/crio-6bbe7a1ec3adee477d0ce058a0bca847096a10a81f4fd5685a8a50a1250e823d WatchSource:0}: Error finding container 6bbe7a1ec3adee477d0ce058a0bca847096a10a81f4fd5685a8a50a1250e823d: Status 404 returned error can't find the container with id 6bbe7a1ec3adee477d0ce058a0bca847096a10a81f4fd5685a8a50a1250e823d Jul 12 18:32:01.305133 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:01.305098 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="713ffc38-dd0f-4f7b-9dbb-5bd17b74295e" path="/var/lib/kubelet/pods/713ffc38-dd0f-4f7b-9dbb-5bd17b74295e/volumes" Jul 12 18:32:01.305591 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:01.305382 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc67d088-2fc9-4b39-b500-25d552959873" path="/var/lib/kubelet/pods/fc67d088-2fc9-4b39-b500-25d552959873/volumes" Jul 12 18:32:01.382562 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:01.382517 2585 generic.go:356] "Generic (PLEG): container finished" podID="fc67d088-2fc9-4b39-b500-25d552959873" containerID="9a9a0f26a0399b29b16c2cbc98cb625604f51c69b0aa59a9c265d6749eee2a67" exitCode=0 Jul 12 18:32:01.382745 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:01.382604 2585 scope.go:117] "RemoveContainer" containerID="9a9a0f26a0399b29b16c2cbc98cb625604f51c69b0aa59a9c265d6749eee2a67" Jul 12 18:32:01.382745 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:01.382604 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsrl9" Jul 12 18:32:01.386805 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:01.386681 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbvzd" event={"ID":"bfb07e88-9932-4ef7-9488-57b79c51578f","Type":"ContainerStarted","Data":"3d8b579d268731246fa32a3fe6fd44a5e91d08ed8fb3ba0d1e317efc703b84b0"} Jul 12 18:32:01.386805 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:01.386741 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbvzd" event={"ID":"bfb07e88-9932-4ef7-9488-57b79c51578f","Type":"ContainerStarted","Data":"6bbe7a1ec3adee477d0ce058a0bca847096a10a81f4fd5685a8a50a1250e823d"} Jul 12 18:32:01.386805 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:01.386793 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbvzd" Jul 12 18:32:01.389355 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:01.389328 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" Jul 12 18:32:01.389355 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:01.389342 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" event={"ID":"7dcb9a46-f43f-4743-96b1-e6fe401fb31d","Type":"ContainerStarted","Data":"d1de023189fcf961a3f38de43ff910e3c301299c26420845b9e16200d7083ef6"} Jul 12 18:32:01.402173 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:01.402145 2585 scope.go:117] "RemoveContainer" containerID="9a9a0f26a0399b29b16c2cbc98cb625604f51c69b0aa59a9c265d6749eee2a67" Jul 12 18:32:01.402514 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:32:01.402484 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a9a0f26a0399b29b16c2cbc98cb625604f51c69b0aa59a9c265d6749eee2a67\": container with ID starting with 9a9a0f26a0399b29b16c2cbc98cb625604f51c69b0aa59a9c265d6749eee2a67 not found: ID does not exist" containerID="9a9a0f26a0399b29b16c2cbc98cb625604f51c69b0aa59a9c265d6749eee2a67" Jul 12 18:32:01.402600 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:01.402529 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a9a0f26a0399b29b16c2cbc98cb625604f51c69b0aa59a9c265d6749eee2a67"} err="failed to get container status \"9a9a0f26a0399b29b16c2cbc98cb625604f51c69b0aa59a9c265d6749eee2a67\": rpc error: code = NotFound desc = could not find container \"9a9a0f26a0399b29b16c2cbc98cb625604f51c69b0aa59a9c265d6749eee2a67\": container with ID starting with 9a9a0f26a0399b29b16c2cbc98cb625604f51c69b0aa59a9c265d6749eee2a67 not found: ID does not exist" Jul 12 18:32:01.411982 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:32:01.411910 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-bxwq6\" is forbidden: User \"system:node:ip-10-0-140-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-170.ec2.internal' and this object" podUID="713ffc38-dd0f-4f7b-9dbb-5bd17b74295e" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-bxwq6" Jul 12 18:32:01.413179 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:01.413115 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbvzd" podStartSLOduration=1.413089482 podStartE2EDuration="1.413089482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:32:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:32:01.410042779 +0000 UTC m=+412.609509814" watchObservedRunningTime="2026-07-12 18:32:01.413089482 +0000 UTC m=+412.612556515" Jul 12 18:32:05.418140 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:05.418102 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" event={"ID":"7dcb9a46-f43f-4743-96b1-e6fe401fb31d","Type":"ContainerStarted","Data":"e409c8af3289678f1b127175e410e0f3ecf5b5108c703bc087e414771232533b"} Jul 12 18:32:05.418530 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:05.418185 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" Jul 12 18:32:05.439857 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:05.439801 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" podStartSLOduration=1.638500462 podStartE2EDuration="5.439784116s" totalImagesPullingTime="3.801283654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:32:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:32:05.436952929 +0000 UTC m=+416.636419949" watchObservedRunningTime="2026-07-12 18:32:05.439784116 +0000 UTC m=+416.639251145" Jul 12 18:32:12.400413 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:12.400383 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kbvzd" Jul 12 18:32:12.647348 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:12.647311 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj"] Jul 12 18:32:12.647548 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:12.647509 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" podUID="7dcb9a46-f43f-4743-96b1-e6fe401fb31d" containerName="manager" containerID="cri-o://e409c8af3289678f1b127175e410e0f3ecf5b5108c703bc087e414771232533b" gracePeriod=10 Jul 12 18:32:12.649430 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:12.649409 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" Jul 12 18:32:12.902942 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:12.902901 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" Jul 12 18:32:13.026538 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.026495 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llhhx\" (UniqueName: \"kubernetes.io/projected/7dcb9a46-f43f-4743-96b1-e6fe401fb31d-kube-api-access-llhhx\") pod \"7dcb9a46-f43f-4743-96b1-e6fe401fb31d\" (UID: \"7dcb9a46-f43f-4743-96b1-e6fe401fb31d\") " Jul 12 18:32:13.026699 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.026557 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7dcb9a46-f43f-4743-96b1-e6fe401fb31d-extensions-socket-volume\") pod \"7dcb9a46-f43f-4743-96b1-e6fe401fb31d\" (UID: \"7dcb9a46-f43f-4743-96b1-e6fe401fb31d\") " Jul 12 18:32:13.027007 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.026984 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dcb9a46-f43f-4743-96b1-e6fe401fb31d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7dcb9a46-f43f-4743-96b1-e6fe401fb31d" (UID: "7dcb9a46-f43f-4743-96b1-e6fe401fb31d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:32:13.028745 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.028713 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dcb9a46-f43f-4743-96b1-e6fe401fb31d-kube-api-access-llhhx" (OuterVolumeSpecName: "kube-api-access-llhhx") pod "7dcb9a46-f43f-4743-96b1-e6fe401fb31d" (UID: "7dcb9a46-f43f-4743-96b1-e6fe401fb31d"). InnerVolumeSpecName "kube-api-access-llhhx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:32:13.127298 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.127258 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7dcb9a46-f43f-4743-96b1-e6fe401fb31d-extensions-socket-volume\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:32:13.127298 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.127294 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llhhx\" (UniqueName: \"kubernetes.io/projected/7dcb9a46-f43f-4743-96b1-e6fe401fb31d-kube-api-access-llhhx\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:32:13.462710 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.462674 2585 generic.go:356] "Generic (PLEG): container finished" podID="7dcb9a46-f43f-4743-96b1-e6fe401fb31d" containerID="e409c8af3289678f1b127175e410e0f3ecf5b5108c703bc087e414771232533b" exitCode=0 Jul 12 18:32:13.463219 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.462724 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" Jul 12 18:32:13.463219 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.462767 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" event={"ID":"7dcb9a46-f43f-4743-96b1-e6fe401fb31d","Type":"ContainerDied","Data":"e409c8af3289678f1b127175e410e0f3ecf5b5108c703bc087e414771232533b"} Jul 12 18:32:13.463219 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.462833 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj" event={"ID":"7dcb9a46-f43f-4743-96b1-e6fe401fb31d","Type":"ContainerDied","Data":"d1de023189fcf961a3f38de43ff910e3c301299c26420845b9e16200d7083ef6"} Jul 12 18:32:13.463219 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.462871 2585 scope.go:117] "RemoveContainer" containerID="e409c8af3289678f1b127175e410e0f3ecf5b5108c703bc087e414771232533b" Jul 12 18:32:13.472450 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.472421 2585 scope.go:117] "RemoveContainer" containerID="e409c8af3289678f1b127175e410e0f3ecf5b5108c703bc087e414771232533b" Jul 12 18:32:13.472823 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:32:13.472771 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e409c8af3289678f1b127175e410e0f3ecf5b5108c703bc087e414771232533b\": container with ID starting with e409c8af3289678f1b127175e410e0f3ecf5b5108c703bc087e414771232533b not found: ID does not exist" containerID="e409c8af3289678f1b127175e410e0f3ecf5b5108c703bc087e414771232533b" Jul 12 18:32:13.472896 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.472833 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e409c8af3289678f1b127175e410e0f3ecf5b5108c703bc087e414771232533b"} err="failed to get container status \"e409c8af3289678f1b127175e410e0f3ecf5b5108c703bc087e414771232533b\": rpc error: code = NotFound desc = could not find container \"e409c8af3289678f1b127175e410e0f3ecf5b5108c703bc087e414771232533b\": container with ID starting with e409c8af3289678f1b127175e410e0f3ecf5b5108c703bc087e414771232533b not found: ID does not exist" Jul 12 18:32:13.499144 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.499107 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj"] Jul 12 18:32:13.513374 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:13.513344 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gxhnj"] Jul 12 18:32:15.304178 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:15.304136 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dcb9a46-f43f-4743-96b1-e6fe401fb31d" path="/var/lib/kubelet/pods/7dcb9a46-f43f-4743-96b1-e6fe401fb31d/volumes" Jul 12 18:32:28.986884 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:28.986845 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns"] Jul 12 18:32:28.987515 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:28.987406 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dcb9a46-f43f-4743-96b1-e6fe401fb31d" containerName="manager" Jul 12 18:32:28.987515 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:28.987427 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dcb9a46-f43f-4743-96b1-e6fe401fb31d" containerName="manager" Jul 12 18:32:28.987626 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:28.987535 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dcb9a46-f43f-4743-96b1-e6fe401fb31d" containerName="manager" Jul 12 18:32:28.994268 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:28.994240 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:28.996979 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:28.996948 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-qvpkp\"" Jul 12 18:32:29.003717 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.003690 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns"] Jul 12 18:32:29.173417 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.173366 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.173417 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.173420 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.173710 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.173487 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc204fa9-67fb-476a-a6f0-c31680438323-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.173710 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.173523 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.173710 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.173538 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.173710 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.173579 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc204fa9-67fb-476a-a6f0-c31680438323-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.173710 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.173629 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k42jm\" (UniqueName: \"kubernetes.io/projected/fc204fa9-67fb-476a-a6f0-c31680438323-kube-api-access-k42jm\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.173710 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.173705 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.173968 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.173742 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc204fa9-67fb-476a-a6f0-c31680438323-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.274407 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.274368 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.274604 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.274410 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.274604 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.274429 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc204fa9-67fb-476a-a6f0-c31680438323-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.274604 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.274562 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k42jm\" (UniqueName: \"kubernetes.io/projected/fc204fa9-67fb-476a-a6f0-c31680438323-kube-api-access-k42jm\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.274845 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.274642 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.274845 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.274679 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc204fa9-67fb-476a-a6f0-c31680438323-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.274845 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.274722 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.274845 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.274758 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.274845 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.274831 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc204fa9-67fb-476a-a6f0-c31680438323-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.275137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.274851 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.275137 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.274877 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.275246 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.275192 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.275246 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.275218 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.275480 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.275463 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc204fa9-67fb-476a-a6f0-c31680438323-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.277340 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.277310 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc204fa9-67fb-476a-a6f0-c31680438323-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.277551 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.277530 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc204fa9-67fb-476a-a6f0-c31680438323-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.284359 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.284320 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc204fa9-67fb-476a-a6f0-c31680438323-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.284475 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.284452 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k42jm\" (UniqueName: \"kubernetes.io/projected/fc204fa9-67fb-476a-a6f0-c31680438323-kube-api-access-k42jm\") pod \"maas-default-gateway-openshift-default-56654dcf7-24lns\" (UID: \"fc204fa9-67fb-476a-a6f0-c31680438323\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.305853 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.305822 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:29.458401 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.458368 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns"] Jul 12 18:32:29.459913 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:32:29.459885 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc204fa9_67fb_476a_a6f0_c31680438323.slice/crio-21a129ffdbf3d711804500c46f392ea6d33745e0641b80f2a4104d13add36c54 WatchSource:0}: Error finding container 21a129ffdbf3d711804500c46f392ea6d33745e0641b80f2a4104d13add36c54: Status 404 returned error can't find the container with id 21a129ffdbf3d711804500c46f392ea6d33745e0641b80f2a4104d13add36c54 Jul 12 18:32:29.461959 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.461910 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:32:29.462018 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.461995 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:32:29.462058 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.462025 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:32:29.551434 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.551398 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" event={"ID":"fc204fa9-67fb-476a-a6f0-c31680438323","Type":"ContainerStarted","Data":"4ce2e2bb7b13072504375e660292d576bfd28ba530b7d5f5be6585b0d1935162"} Jul 12 18:32:29.551599 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.551466 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" event={"ID":"fc204fa9-67fb-476a-a6f0-c31680438323","Type":"ContainerStarted","Data":"21a129ffdbf3d711804500c46f392ea6d33745e0641b80f2a4104d13add36c54"} Jul 12 18:32:29.589819 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:29.589753 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" podStartSLOduration=1.5897333759999999 podStartE2EDuration="1.589733376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:32:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:32:29.587614713 +0000 UTC m=+440.787081747" watchObservedRunningTime="2026-07-12 18:32:29.589733376 +0000 UTC m=+440.789200407" Jul 12 18:32:30.306922 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:30.306882 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:30.312452 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:30.312421 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:30.556629 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:30.556597 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:30.557809 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:30.557731 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:32:30.952465 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:30.952365 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:32:30.956161 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:30.956137 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-45x44" Jul 12 18:32:30.958426 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:30.958399 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 12 18:32:30.958555 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:30.958451 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2dvvh\"" Jul 12 18:32:30.962986 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:30.962961 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:32:30.986742 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:30.986697 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:32:31.094782 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:31.094724 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d9455fd6-e2d9-4886-9be8-c554096d4e33-config-file\") pod \"limitador-limitador-54d6d988d5-45x44\" (UID: \"d9455fd6-e2d9-4886-9be8-c554096d4e33\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-45x44" Jul 12 18:32:31.095011 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:31.094870 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmhfb\" (UniqueName: \"kubernetes.io/projected/d9455fd6-e2d9-4886-9be8-c554096d4e33-kube-api-access-qmhfb\") pod \"limitador-limitador-54d6d988d5-45x44\" (UID: \"d9455fd6-e2d9-4886-9be8-c554096d4e33\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-45x44" Jul 12 18:32:31.195455 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:31.195396 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmhfb\" (UniqueName: \"kubernetes.io/projected/d9455fd6-e2d9-4886-9be8-c554096d4e33-kube-api-access-qmhfb\") pod \"limitador-limitador-54d6d988d5-45x44\" (UID: \"d9455fd6-e2d9-4886-9be8-c554096d4e33\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-45x44" Jul 12 18:32:31.195638 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:31.195488 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d9455fd6-e2d9-4886-9be8-c554096d4e33-config-file\") pod \"limitador-limitador-54d6d988d5-45x44\" (UID: \"d9455fd6-e2d9-4886-9be8-c554096d4e33\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-45x44" Jul 12 18:32:31.196168 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:31.196148 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d9455fd6-e2d9-4886-9be8-c554096d4e33-config-file\") pod \"limitador-limitador-54d6d988d5-45x44\" (UID: \"d9455fd6-e2d9-4886-9be8-c554096d4e33\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-45x44" Jul 12 18:32:31.204377 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:31.204303 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmhfb\" (UniqueName: \"kubernetes.io/projected/d9455fd6-e2d9-4886-9be8-c554096d4e33-kube-api-access-qmhfb\") pod \"limitador-limitador-54d6d988d5-45x44\" (UID: \"d9455fd6-e2d9-4886-9be8-c554096d4e33\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-45x44" Jul 12 18:32:31.268795 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:31.268746 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-45x44" Jul 12 18:32:31.412728 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:31.412705 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:32:31.415484 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:32:31.415445 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9455fd6_e2d9_4886_9be8_c554096d4e33.slice/crio-d03a182ffcd43386f6261b5d08cf7725689724b44f2a599bf3b031253e3d3079 WatchSource:0}: Error finding container d03a182ffcd43386f6261b5d08cf7725689724b44f2a599bf3b031253e3d3079: Status 404 returned error can't find the container with id d03a182ffcd43386f6261b5d08cf7725689724b44f2a599bf3b031253e3d3079 Jul 12 18:32:31.562824 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:31.562785 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-45x44" event={"ID":"d9455fd6-e2d9-4886-9be8-c554096d4e33","Type":"ContainerStarted","Data":"d03a182ffcd43386f6261b5d08cf7725689724b44f2a599bf3b031253e3d3079"} Jul 12 18:32:34.582149 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:34.582100 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-45x44" event={"ID":"d9455fd6-e2d9-4886-9be8-c554096d4e33","Type":"ContainerStarted","Data":"af2ee13924467dcb94f8fd8be16b2c9bd6ab77361b6f21a92d4a5ae2b3c18f5e"} Jul 12 18:32:34.582655 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:34.582236 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-45x44" Jul 12 18:32:34.600710 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:34.600647 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-45x44" podStartSLOduration=2.038761881 podStartE2EDuration="4.600627043s" totalImagesPullingTime="2.561865162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:32:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:32:34.598551682 +0000 UTC m=+445.798018712" watchObservedRunningTime="2026-07-12 18:32:34.600627043 +0000 UTC m=+445.800094072" Jul 12 18:32:45.588307 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:32:45.588272 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-45x44" Jul 12 18:33:45.673058 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:45.673013 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:33:54.010456 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:54.010418 2585 generic.go:356] "Generic (PLEG): container finished" podID="fc204fa9-67fb-476a-a6f0-c31680438323" containerID="4ce2e2bb7b13072504375e660292d576bfd28ba530b7d5f5be6585b0d1935162" exitCode=0 Jul 12 18:33:54.010915 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:54.010491 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" event={"ID":"fc204fa9-67fb-476a-a6f0-c31680438323","Type":"ContainerDied","Data":"4ce2e2bb7b13072504375e660292d576bfd28ba530b7d5f5be6585b0d1935162"} Jul 12 18:33:54.010915 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:54.010897 2585 scope.go:117] "RemoveContainer" containerID="4ce2e2bb7b13072504375e660292d576bfd28ba530b7d5f5be6585b0d1935162" Jul 12 18:33:54.012004 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:54.011974 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:33:54.012098 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:54.012052 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:33:54.012098 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:54.012087 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:33:55.018124 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:55.018092 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" event={"ID":"fc204fa9-67fb-476a-a6f0-c31680438323","Type":"ContainerStarted","Data":"ddc2d4317a30d0b7098258d74613c06897df5c16fe47f537f7cb25ab46c5e8a2"} Jul 12 18:33:55.306055 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:55.305977 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:33:55.306232 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:55.306201 2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-24lns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 12 18:33:55.306284 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:55.306260 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" podUID="fc204fa9-67fb-476a-a6f0-c31680438323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 12 18:33:56.306266 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:56.306221 2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-24lns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 12 18:33:56.306756 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:56.306286 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" podUID="fc204fa9-67fb-476a-a6f0-c31680438323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 12 18:33:57.306206 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:57.306169 2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-24lns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 12 18:33:57.306387 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:57.306229 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" podUID="fc204fa9-67fb-476a-a6f0-c31680438323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 12 18:33:58.306779 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:58.306739 2585 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-24lns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 12 18:33:58.307227 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:58.306803 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" podUID="fc204fa9-67fb-476a-a6f0-c31680438323" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 12 18:33:59.310844 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:59.310815 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:33:59.311259 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:59.311144 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:33:59.312114 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:33:59.312097 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-24lns" Jul 12 18:34:02.823487 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:02.823404 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:34:11.693359 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.693317 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8"] Jul 12 18:34:11.696634 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.696611 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.699241 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.699211 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 12 18:34:11.700069 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.700049 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 12 18:34:11.700226 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.700048 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6qzs6\"" Jul 12 18:34:11.700226 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.700054 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 12 18:34:11.705273 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.705243 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8"] Jul 12 18:34:11.799815 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.799768 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58zn9\" (UniqueName: \"kubernetes.io/projected/6d60474b-9767-4eb2-892d-5906ff231fea-kube-api-access-58zn9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.800033 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.799836 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d60474b-9767-4eb2-892d-5906ff231fea-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.800033 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.799880 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d60474b-9767-4eb2-892d-5906ff231fea-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.800033 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.799898 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d60474b-9767-4eb2-892d-5906ff231fea-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.800033 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.800004 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d60474b-9767-4eb2-892d-5906ff231fea-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.800171 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.800044 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d60474b-9767-4eb2-892d-5906ff231fea-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.900705 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.900657 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d60474b-9767-4eb2-892d-5906ff231fea-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.900705 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.900705 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d60474b-9767-4eb2-892d-5906ff231fea-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.900983 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.900748 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d60474b-9767-4eb2-892d-5906ff231fea-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.900983 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.900769 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d60474b-9767-4eb2-892d-5906ff231fea-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.900983 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.900789 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58zn9\" (UniqueName: \"kubernetes.io/projected/6d60474b-9767-4eb2-892d-5906ff231fea-kube-api-access-58zn9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.900983 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.900834 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d60474b-9767-4eb2-892d-5906ff231fea-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.901221 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.901198 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d60474b-9767-4eb2-892d-5906ff231fea-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.901285 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.901242 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d60474b-9767-4eb2-892d-5906ff231fea-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.901339 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.901285 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d60474b-9767-4eb2-892d-5906ff231fea-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.903253 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.903229 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d60474b-9767-4eb2-892d-5906ff231fea-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.903403 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.903386 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d60474b-9767-4eb2-892d-5906ff231fea-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:11.909481 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:11.909446 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58zn9\" (UniqueName: \"kubernetes.io/projected/6d60474b-9767-4eb2-892d-5906ff231fea-kube-api-access-58zn9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jwjb8\" (UID: \"6d60474b-9767-4eb2-892d-5906ff231fea\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:12.012070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:12.011971 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:12.157262 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:12.157232 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8"] Jul 12 18:34:12.159185 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:34:12.159153 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d60474b_9767_4eb2_892d_5906ff231fea.slice/crio-0ded786c1c892dfa5e160997adf5a3bc907229d8252b7aa3ba54b8dff9f85dd7 WatchSource:0}: Error finding container 0ded786c1c892dfa5e160997adf5a3bc907229d8252b7aa3ba54b8dff9f85dd7: Status 404 returned error can't find the container with id 0ded786c1c892dfa5e160997adf5a3bc907229d8252b7aa3ba54b8dff9f85dd7 Jul 12 18:34:12.306282 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:12.306248 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:34:13.130282 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:13.130248 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" event={"ID":"6d60474b-9767-4eb2-892d-5906ff231fea","Type":"ContainerStarted","Data":"0ded786c1c892dfa5e160997adf5a3bc907229d8252b7aa3ba54b8dff9f85dd7"} Jul 12 18:34:14.137338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:14.137300 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" event={"ID":"6d60474b-9767-4eb2-892d-5906ff231fea","Type":"ContainerStarted","Data":"6801735ecdb6bf850b4dfaaf295594692f0cc44d5ad987c0d34806b2dc73323a"} Jul 12 18:34:14.158077 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:14.158022 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" podStartSLOduration=2.201631272 podStartE2EDuration="3.158005336s" totalImagesPullingTime="956.374064ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:34:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:34:14.156260554 +0000 UTC m=+545.355727585" watchObservedRunningTime="2026-07-12 18:34:14.158005336 +0000 UTC m=+545.357472411" Jul 12 18:34:22.012421 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:22.012379 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:22.012982 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:22.012459 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:22.025816 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:22.025781 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:22.191978 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:22.191914 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jwjb8" Jul 12 18:34:23.714772 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:23.714730 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:34:40.808303 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:34:40.808265 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:35:05.012707 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:35:05.012670 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:35:09.283534 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:35:09.283508 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:35:09.284033 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:35:09.283886 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:35:22.009105 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:35:22.009067 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:36:52.510896 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:36:52.510853 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:37:03.914669 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:37:03.914577 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:37:14.510453 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:37:14.510408 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:37:24.410705 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:37:24.410670 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:37:34.706578 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:37:34.706535 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:38:02.019446 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:38:02.019406 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:38:14.406491 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:38:14.406448 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:38:50.210337 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:38:50.210296 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:39:01.507719 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:39:01.507679 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:39:14.111252 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:39:14.111213 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:39:24.012977 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:39:24.012925 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:39:35.813632 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:39:35.813589 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:39:47.215671 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:39:47.215632 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:39:59.709795 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:39:59.709751 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:40:08.415526 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:40:08.415437 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:40:09.318146 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:40:09.318117 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:40:09.319969 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:40:09.319943 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:40:33.513515 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:40:33.513474 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:40:46.206672 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:40:46.206631 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:41:06.810369 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:41:06.810328 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:41:18.102276 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:41:18.102236 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:41:33.914022 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:41:33.913914 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:41:43.505117 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:41:43.505066 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:41:56.510997 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:41:56.510959 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:44:02.903300 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:44:02.903258 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:44:17.114489 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:44:17.114450 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:44:26.113773 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:44:26.110442 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:44:39.108732 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:44:39.108637 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:44:50.704565 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:44:50.704522 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:45:00.141249 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:00.141207 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8"] Jul 12 18:45:00.145130 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:00.145104 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8" Jul 12 18:45:00.147478 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:00.147451 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 12 18:45:00.147617 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:00.147508 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 12 18:45:00.147617 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:00.147525 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-j8nzl\"" Jul 12 18:45:00.151767 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:00.151744 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8"] Jul 12 18:45:00.253686 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:00.253641 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdglv\" (UniqueName: \"kubernetes.io/projected/6cefdc98-2be1-4fdb-aaef-554890f6a4f7-kube-api-access-tdglv\") pod \"maas-api-key-cleanup-29731365-dx2n8\" (UID: \"6cefdc98-2be1-4fdb-aaef-554890f6a4f7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8" Jul 12 18:45:00.354151 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:00.354106 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdglv\" (UniqueName: \"kubernetes.io/projected/6cefdc98-2be1-4fdb-aaef-554890f6a4f7-kube-api-access-tdglv\") pod \"maas-api-key-cleanup-29731365-dx2n8\" (UID: \"6cefdc98-2be1-4fdb-aaef-554890f6a4f7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8" Jul 12 18:45:00.363641 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:00.363607 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdglv\" (UniqueName: \"kubernetes.io/projected/6cefdc98-2be1-4fdb-aaef-554890f6a4f7-kube-api-access-tdglv\") pod \"maas-api-key-cleanup-29731365-dx2n8\" (UID: \"6cefdc98-2be1-4fdb-aaef-554890f6a4f7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8" Jul 12 18:45:00.458067 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:00.457975 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8" Jul 12 18:45:00.596780 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:00.596749 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8"] Jul 12 18:45:00.599115 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:45:00.599084 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cefdc98_2be1_4fdb_aaef_554890f6a4f7.slice/crio-39b2a053078371f2b55f287ee71989d03bd416e432f69cec8d91ea83dd9839d9 WatchSource:0}: Error finding container 39b2a053078371f2b55f287ee71989d03bd416e432f69cec8d91ea83dd9839d9: Status 404 returned error can't find the container with id 39b2a053078371f2b55f287ee71989d03bd416e432f69cec8d91ea83dd9839d9 Jul 12 18:45:00.600904 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:00.600886 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 18:45:00.639075 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:00.639037 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8" event={"ID":"6cefdc98-2be1-4fdb-aaef-554890f6a4f7","Type":"ContainerStarted","Data":"39b2a053078371f2b55f287ee71989d03bd416e432f69cec8d91ea83dd9839d9"} Jul 12 18:45:02.002474 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:02.002432 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:45:04.664556 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:04.664519 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8" event={"ID":"6cefdc98-2be1-4fdb-aaef-554890f6a4f7","Type":"ContainerStarted","Data":"f56a7aeba0f34f6acd904bd9a61635e6e2df4feb4b5360b5f9e6fead35196831"} Jul 12 18:45:04.682274 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:04.682218 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8" podStartSLOduration=1.86061733 podStartE2EDuration="4.68220013s" totalImagesPullingTime="2.8215828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:45:04.680785829 +0000 UTC m=+1195.880252879" watchObservedRunningTime="2026-07-12 18:45:04.68220013 +0000 UTC m=+1195.881667254" Jul 12 18:45:09.358178 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:09.358146 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:45:09.358645 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:09.358585 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:45:11.617864 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:11.617820 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:45:23.911361 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:23.911310 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:45:41.112560 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:41.112509 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:45:53.712106 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:45:53.712066 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:46:00.011108 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:00.011068 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8"] Jul 12 18:46:00.011512 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:00.011274 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8" podUID="6cefdc98-2be1-4fdb-aaef-554890f6a4f7" containerName="cleanup" containerID="cri-o://f56a7aeba0f34f6acd904bd9a61635e6e2df4feb4b5360b5f9e6fead35196831" gracePeriod=30 Jul 12 18:46:16.316187 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:16.316098 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:46:27.568151 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:27.568101 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:46:30.153571 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:30.153538 2585 generic.go:356] "Generic (PLEG): container finished" podID="6cefdc98-2be1-4fdb-aaef-554890f6a4f7" containerID="f56a7aeba0f34f6acd904bd9a61635e6e2df4feb4b5360b5f9e6fead35196831" exitCode=137 Jul 12 18:46:30.153960 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:30.153612 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8" event={"ID":"6cefdc98-2be1-4fdb-aaef-554890f6a4f7","Type":"ContainerDied","Data":"f56a7aeba0f34f6acd904bd9a61635e6e2df4feb4b5360b5f9e6fead35196831"} Jul 12 18:46:30.209206 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:30.209177 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8" Jul 12 18:46:30.358445 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:30.358353 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdglv\" (UniqueName: \"kubernetes.io/projected/6cefdc98-2be1-4fdb-aaef-554890f6a4f7-kube-api-access-tdglv\") pod \"6cefdc98-2be1-4fdb-aaef-554890f6a4f7\" (UID: \"6cefdc98-2be1-4fdb-aaef-554890f6a4f7\") " Jul 12 18:46:30.360582 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:30.360547 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cefdc98-2be1-4fdb-aaef-554890f6a4f7-kube-api-access-tdglv" (OuterVolumeSpecName: "kube-api-access-tdglv") pod "6cefdc98-2be1-4fdb-aaef-554890f6a4f7" (UID: "6cefdc98-2be1-4fdb-aaef-554890f6a4f7"). InnerVolumeSpecName "kube-api-access-tdglv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:46:30.459850 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:30.459813 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdglv\" (UniqueName: \"kubernetes.io/projected/6cefdc98-2be1-4fdb-aaef-554890f6a4f7-kube-api-access-tdglv\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:46:31.160924 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:31.160889 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8" Jul 12 18:46:31.160924 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:31.160904 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8" event={"ID":"6cefdc98-2be1-4fdb-aaef-554890f6a4f7","Type":"ContainerDied","Data":"39b2a053078371f2b55f287ee71989d03bd416e432f69cec8d91ea83dd9839d9"} Jul 12 18:46:31.161484 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:31.161014 2585 scope.go:117] "RemoveContainer" containerID="f56a7aeba0f34f6acd904bd9a61635e6e2df4feb4b5360b5f9e6fead35196831" Jul 12 18:46:31.182922 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:31.182886 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8"] Jul 12 18:46:31.185180 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:31.185151 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29731365-dx2n8"] Jul 12 18:46:31.304844 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:31.304797 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cefdc98-2be1-4fdb-aaef-554890f6a4f7" path="/var/lib/kubelet/pods/6cefdc98-2be1-4fdb-aaef-554890f6a4f7/volumes" Jul 12 18:46:38.611249 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:38.611209 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:46:50.710643 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:46:50.710599 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:47:01.209905 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:47:01.209865 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:47:10.111574 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:47:10.111533 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:47:23.612736 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:47:23.612697 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:47:34.431315 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:47:34.431226 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:47:53.002507 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:47:53.002466 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:48:17.916327 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:48:17.916285 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:48:33.606744 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:48:33.606705 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:48:47.210193 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:48:47.210154 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:48:57.112469 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:48:57.112429 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:49:07.808740 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:49:07.808690 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:49:22.512465 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:49:22.512422 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:49:38.313863 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:49:38.313820 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:49:49.213966 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:49:49.213906 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:50:05.015997 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:05.015947 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:50:09.393198 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:09.393169 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:50:09.395989 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:09.395965 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:50:10.542571 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.542534 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j"] Jul 12 18:50:10.542984 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.542896 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cefdc98-2be1-4fdb-aaef-554890f6a4f7" containerName="cleanup" Jul 12 18:50:10.542984 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.542906 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cefdc98-2be1-4fdb-aaef-554890f6a4f7" containerName="cleanup" Jul 12 18:50:10.543070 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.542987 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cefdc98-2be1-4fdb-aaef-554890f6a4f7" containerName="cleanup" Jul 12 18:50:10.546117 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.546094 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.548765 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.548740 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-2953509d-openshift-default-dockercfg-bs8ls\"" Jul 12 18:50:10.567602 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.567570 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j"] Jul 12 18:50:10.671261 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.671225 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-workload-certs\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.671261 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.671262 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-data\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.671533 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.671293 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-credential-socket\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.671533 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.671335 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-podinfo\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.671533 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.671370 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istiod-ca-cert\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.671533 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.671401 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-token\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.671533 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.671463 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-envoy\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.671718 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.671549 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9w7t\" (UniqueName: \"kubernetes.io/projected/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-kube-api-access-b9w7t\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.671718 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.671586 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-workload-socket\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.772740 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.772703 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-envoy\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.773003 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.772780 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9w7t\" (UniqueName: \"kubernetes.io/projected/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-kube-api-access-b9w7t\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.773003 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.772803 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-workload-socket\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.773003 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.772840 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-workload-certs\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.773003 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.772859 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-data\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.773003 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.772908 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-credential-socket\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.773003 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.772965 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-podinfo\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.773003 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.772988 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istiod-ca-cert\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.773366 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.773021 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-token\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.773366 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.773283 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-workload-certs\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.773366 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.773316 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-workload-socket\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.773490 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.773474 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-credential-socket\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.773525 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.773500 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-data\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.774016 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.773993 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istiod-ca-cert\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.775075 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.775057 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-envoy\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.775482 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.775464 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-podinfo\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.780878 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.780850 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-token\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.781038 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.780921 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9w7t\" (UniqueName: \"kubernetes.io/projected/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-kube-api-access-b9w7t\") pod \"e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:10.857245 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:10.857159 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:11.008604 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:11.008571 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j"] Jul 12 18:50:11.010024 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:50:11.009995 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea645ef2_5735_4a68_a6b0_a6a1b1f6548f.slice/crio-9c27bb330deaed1f0c441d95c677c63b403dee75dc33fb8911b36d440e4461b4 WatchSource:0}: Error finding container 9c27bb330deaed1f0c441d95c677c63b403dee75dc33fb8911b36d440e4461b4: Status 404 returned error can't find the container with id 9c27bb330deaed1f0c441d95c677c63b403dee75dc33fb8911b36d440e4461b4 Jul 12 18:50:11.012204 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:11.012165 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:50:11.012319 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:11.012273 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:50:11.012382 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:11.012332 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:50:11.355878 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:11.355842 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" event={"ID":"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f","Type":"ContainerStarted","Data":"9dfa2f71a46f6dc060d10c37db0cd2e51263306b9af6821c8c635cbcc729f270"} Jul 12 18:50:11.356115 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:11.355900 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" event={"ID":"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f","Type":"ContainerStarted","Data":"9c27bb330deaed1f0c441d95c677c63b403dee75dc33fb8911b36d440e4461b4"} Jul 12 18:50:11.375923 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:11.375842 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" podStartSLOduration=1.375825552 podStartE2EDuration="1.375825552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:50:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:50:11.374169039 +0000 UTC m=+1502.573636071" watchObservedRunningTime="2026-07-12 18:50:11.375825552 +0000 UTC m=+1502.575292581" Jul 12 18:50:11.857728 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:11.857683 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:12.863305 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:12.863273 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:13.367196 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:13.367155 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:13.368221 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:13.368200 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:16.925881 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:16.925829 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j"] Jul 12 18:50:16.926723 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:16.926684 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" podUID="ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" containerName="istio-proxy" containerID="cri-o://9dfa2f71a46f6dc060d10c37db0cd2e51263306b9af6821c8c635cbcc729f270" gracePeriod=30 Jul 12 18:50:17.016257 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:17.016219 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:50:22.189072 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.189007 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:22.279072 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279033 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istiod-ca-cert\") pod \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " Jul 12 18:50:22.279072 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279073 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-workload-socket\") pod \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " Jul 12 18:50:22.279304 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279092 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-envoy\") pod \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " Jul 12 18:50:22.279304 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279116 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9w7t\" (UniqueName: \"kubernetes.io/projected/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-kube-api-access-b9w7t\") pod \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " Jul 12 18:50:22.279304 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279165 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-podinfo\") pod \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " Jul 12 18:50:22.279304 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279236 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-data\") pod \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " Jul 12 18:50:22.279304 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279289 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-token\") pod \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " Jul 12 18:50:22.279574 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279334 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-workload-certs\") pod \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " Jul 12 18:50:22.279574 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279352 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-credential-socket\") pod \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\" (UID: \"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f\") " Jul 12 18:50:22.279574 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279453 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" (UID: "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:22.279574 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279460 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" (UID: "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:50:22.279787 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279645 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istiod-ca-cert\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:22.279787 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279664 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-workload-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:22.279787 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279671 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" (UID: "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:22.279787 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279726 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" (UID: "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:22.280051 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.279817 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-data" (OuterVolumeSpecName: "istio-data") pod "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" (UID: "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:22.281594 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.281566 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" (UID: "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:22.281723 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.281632 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-kube-api-access-b9w7t" (OuterVolumeSpecName: "kube-api-access-b9w7t") pod "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" (UID: "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f"). InnerVolumeSpecName "kube-api-access-b9w7t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:50:22.281723 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.281657 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-token" (OuterVolumeSpecName: "istio-token") pod "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" (UID: "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:50:22.281723 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.281684 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" (UID: "ea645ef2-5735-4a68-a6b0-a6a1b1f6548f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:50:22.380358 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.380321 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9w7t\" (UniqueName: \"kubernetes.io/projected/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-kube-api-access-b9w7t\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:22.380358 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.380351 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-podinfo\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:22.380358 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.380361 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-data\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:22.380358 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.380369 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-token\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:22.380624 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.380378 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-workload-certs\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:22.380624 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.380386 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-credential-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:22.380624 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.380393 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f-istio-envoy\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:22.415849 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.415815 2585 generic.go:356] "Generic (PLEG): container finished" podID="ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" containerID="9dfa2f71a46f6dc060d10c37db0cd2e51263306b9af6821c8c635cbcc729f270" exitCode=0 Jul 12 18:50:22.416069 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.415859 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" Jul 12 18:50:22.416069 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.415860 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" event={"ID":"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f","Type":"ContainerDied","Data":"9dfa2f71a46f6dc060d10c37db0cd2e51263306b9af6821c8c635cbcc729f270"} Jul 12 18:50:22.416069 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.416005 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j" event={"ID":"ea645ef2-5735-4a68-a6b0-a6a1b1f6548f","Type":"ContainerDied","Data":"9c27bb330deaed1f0c441d95c677c63b403dee75dc33fb8911b36d440e4461b4"} Jul 12 18:50:22.416069 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.416044 2585 scope.go:117] "RemoveContainer" containerID="9dfa2f71a46f6dc060d10c37db0cd2e51263306b9af6821c8c635cbcc729f270" Jul 12 18:50:22.425798 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.425780 2585 scope.go:117] "RemoveContainer" containerID="9dfa2f71a46f6dc060d10c37db0cd2e51263306b9af6821c8c635cbcc729f270" Jul 12 18:50:22.426132 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:50:22.426112 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dfa2f71a46f6dc060d10c37db0cd2e51263306b9af6821c8c635cbcc729f270\": container with ID starting with 9dfa2f71a46f6dc060d10c37db0cd2e51263306b9af6821c8c635cbcc729f270 not found: ID does not exist" containerID="9dfa2f71a46f6dc060d10c37db0cd2e51263306b9af6821c8c635cbcc729f270" Jul 12 18:50:22.426184 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.426142 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dfa2f71a46f6dc060d10c37db0cd2e51263306b9af6821c8c635cbcc729f270"} err="failed to get container status \"9dfa2f71a46f6dc060d10c37db0cd2e51263306b9af6821c8c635cbcc729f270\": rpc error: code = NotFound desc = could not find container \"9dfa2f71a46f6dc060d10c37db0cd2e51263306b9af6821c8c635cbcc729f270\": container with ID starting with 9dfa2f71a46f6dc060d10c37db0cd2e51263306b9af6821c8c635cbcc729f270 not found: ID does not exist" Jul 12 18:50:22.437185 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.437153 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j"] Jul 12 18:50:22.446146 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.446020 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-2953509d-openshift-default-6bfc88b6b5-v847j"] Jul 12 18:50:22.801099 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.801059 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954"] Jul 12 18:50:22.801667 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.801648 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" containerName="istio-proxy" Jul 12 18:50:22.801667 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.801669 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" containerName="istio-proxy" Jul 12 18:50:22.801854 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.801803 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" containerName="istio-proxy" Jul 12 18:50:22.805350 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.805319 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:22.807677 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.807652 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-73e47abc-openshift-default-dockercfg-gx5dj\"" Jul 12 18:50:22.816576 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.816546 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954"] Jul 12 18:50:22.984711 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.984669 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-data\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:22.984877 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.984743 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-workload-socket\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:22.984877 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.984767 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-envoy\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:22.984877 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.984783 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsqcs\" (UniqueName: \"kubernetes.io/projected/d109fe04-a8c7-4e05-abc2-ac27e0293f60-kube-api-access-qsqcs\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:22.985065 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.984864 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-workload-certs\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:22.985065 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.984898 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-podinfo\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:22.985065 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.984914 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-token\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:22.985065 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.984966 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-credential-socket\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:22.985065 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:22.985000 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istiod-ca-cert\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.085451 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.085370 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-workload-certs\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.085451 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.085403 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-podinfo\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.085451 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.085419 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-token\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.085451 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.085443 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-credential-socket\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.085791 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.085470 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istiod-ca-cert\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.085791 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.085504 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-data\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.085791 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.085547 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-workload-socket\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.085791 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.085621 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-envoy\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.085791 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.085667 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsqcs\" (UniqueName: \"kubernetes.io/projected/d109fe04-a8c7-4e05-abc2-ac27e0293f60-kube-api-access-qsqcs\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.086095 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.085896 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-workload-certs\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.086095 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.085978 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-data\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.086095 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.086056 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-workload-socket\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.086305 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.086284 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-credential-socket\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.086346 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.086328 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istiod-ca-cert\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.088014 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.087989 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-envoy\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.088124 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.088045 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-podinfo\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.094238 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.094215 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-token\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.094359 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.094339 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsqcs\" (UniqueName: \"kubernetes.io/projected/d109fe04-a8c7-4e05-abc2-ac27e0293f60-kube-api-access-qsqcs\") pod \"e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.118516 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.118477 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:23.263720 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.263692 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954"] Jul 12 18:50:23.265235 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:50:23.265209 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd109fe04_a8c7_4e05_abc2_ac27e0293f60.slice/crio-6127221f97126d6acf545ae222cfacd878e376922cc76e05bfb3f4150f298c88 WatchSource:0}: Error finding container 6127221f97126d6acf545ae222cfacd878e376922cc76e05bfb3f4150f298c88: Status 404 returned error can't find the container with id 6127221f97126d6acf545ae222cfacd878e376922cc76e05bfb3f4150f298c88 Jul 12 18:50:23.267607 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.267574 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:50:23.267678 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.267644 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:50:23.267713 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.267675 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:50:23.305499 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.305462 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea645ef2-5735-4a68-a6b0-a6a1b1f6548f" path="/var/lib/kubelet/pods/ea645ef2-5735-4a68-a6b0-a6a1b1f6548f/volumes" Jul 12 18:50:23.425623 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.425519 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" event={"ID":"d109fe04-a8c7-4e05-abc2-ac27e0293f60","Type":"ContainerStarted","Data":"dec90ebde6f12d93c4b6edb0230a83aaa3c9288ec1d0f9dee13f090e0b785a97"} Jul 12 18:50:23.425623 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.425587 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" event={"ID":"d109fe04-a8c7-4e05-abc2-ac27e0293f60","Type":"ContainerStarted","Data":"6127221f97126d6acf545ae222cfacd878e376922cc76e05bfb3f4150f298c88"} Jul 12 18:50:23.445668 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:23.445610 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" podStartSLOduration=1.445594282 podStartE2EDuration="1.445594282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:50:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:50:23.44398657 +0000 UTC m=+1514.643453601" watchObservedRunningTime="2026-07-12 18:50:23.445594282 +0000 UTC m=+1514.645061312" Jul 12 18:50:24.119089 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:24.119051 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:24.124841 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:24.124814 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:24.430831 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:24.430751 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:24.431729 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:24.431708 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:27.507325 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:27.507288 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:50:35.060054 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.060015 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954"] Jul 12 18:50:35.060439 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.060234 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" podUID="d109fe04-a8c7-4e05-abc2-ac27e0293f60" containerName="istio-proxy" containerID="cri-o://dec90ebde6f12d93c4b6edb0230a83aaa3c9288ec1d0f9dee13f090e0b785a97" gracePeriod=30 Jul 12 18:50:35.420514 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.420489 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:35.493456 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.493420 2585 generic.go:356] "Generic (PLEG): container finished" podID="d109fe04-a8c7-4e05-abc2-ac27e0293f60" containerID="dec90ebde6f12d93c4b6edb0230a83aaa3c9288ec1d0f9dee13f090e0b785a97" exitCode=0 Jul 12 18:50:35.493654 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.493465 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" Jul 12 18:50:35.493654 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.493495 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" event={"ID":"d109fe04-a8c7-4e05-abc2-ac27e0293f60","Type":"ContainerDied","Data":"dec90ebde6f12d93c4b6edb0230a83aaa3c9288ec1d0f9dee13f090e0b785a97"} Jul 12 18:50:35.493654 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.493550 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954" event={"ID":"d109fe04-a8c7-4e05-abc2-ac27e0293f60","Type":"ContainerDied","Data":"6127221f97126d6acf545ae222cfacd878e376922cc76e05bfb3f4150f298c88"} Jul 12 18:50:35.493654 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.493574 2585 scope.go:117] "RemoveContainer" containerID="dec90ebde6f12d93c4b6edb0230a83aaa3c9288ec1d0f9dee13f090e0b785a97" Jul 12 18:50:35.503880 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.503848 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-data\") pod \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " Jul 12 18:50:35.504036 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.503909 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istiod-ca-cert\") pod \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " Jul 12 18:50:35.504036 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.503955 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-envoy\") pod \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " Jul 12 18:50:35.504036 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.503974 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-podinfo\") pod \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " Jul 12 18:50:35.504036 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.503993 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-token\") pod \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " Jul 12 18:50:35.504220 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.504038 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-workload-certs\") pod \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " Jul 12 18:50:35.504220 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.504065 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-credential-socket\") pod \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " Jul 12 18:50:35.504220 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.504110 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-workload-socket\") pod \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " Jul 12 18:50:35.504220 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.504185 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsqcs\" (UniqueName: \"kubernetes.io/projected/d109fe04-a8c7-4e05-abc2-ac27e0293f60-kube-api-access-qsqcs\") pod \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\" (UID: \"d109fe04-a8c7-4e05-abc2-ac27e0293f60\") " Jul 12 18:50:35.504421 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.504368 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d109fe04-a8c7-4e05-abc2-ac27e0293f60" (UID: "d109fe04-a8c7-4e05-abc2-ac27e0293f60"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:50:35.504515 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.504495 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d109fe04-a8c7-4e05-abc2-ac27e0293f60" (UID: "d109fe04-a8c7-4e05-abc2-ac27e0293f60"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:35.504674 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.504624 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istiod-ca-cert\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:35.504742 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.504715 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-data" (OuterVolumeSpecName: "istio-data") pod "d109fe04-a8c7-4e05-abc2-ac27e0293f60" (UID: "d109fe04-a8c7-4e05-abc2-ac27e0293f60"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:35.504840 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.504822 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d109fe04-a8c7-4e05-abc2-ac27e0293f60" (UID: "d109fe04-a8c7-4e05-abc2-ac27e0293f60"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:35.505086 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.505062 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d109fe04-a8c7-4e05-abc2-ac27e0293f60" (UID: "d109fe04-a8c7-4e05-abc2-ac27e0293f60"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:35.505815 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.505793 2585 scope.go:117] "RemoveContainer" containerID="dec90ebde6f12d93c4b6edb0230a83aaa3c9288ec1d0f9dee13f090e0b785a97" Jul 12 18:50:35.506404 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:50:35.506376 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dec90ebde6f12d93c4b6edb0230a83aaa3c9288ec1d0f9dee13f090e0b785a97\": container with ID starting with dec90ebde6f12d93c4b6edb0230a83aaa3c9288ec1d0f9dee13f090e0b785a97 not found: ID does not exist" containerID="dec90ebde6f12d93c4b6edb0230a83aaa3c9288ec1d0f9dee13f090e0b785a97" Jul 12 18:50:35.506504 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.506418 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dec90ebde6f12d93c4b6edb0230a83aaa3c9288ec1d0f9dee13f090e0b785a97"} err="failed to get container status \"dec90ebde6f12d93c4b6edb0230a83aaa3c9288ec1d0f9dee13f090e0b785a97\": rpc error: code = NotFound desc = could not find container \"dec90ebde6f12d93c4b6edb0230a83aaa3c9288ec1d0f9dee13f090e0b785a97\": container with ID starting with dec90ebde6f12d93c4b6edb0230a83aaa3c9288ec1d0f9dee13f090e0b785a97 not found: ID does not exist" Jul 12 18:50:35.506797 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.506773 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d109fe04-a8c7-4e05-abc2-ac27e0293f60" (UID: "d109fe04-a8c7-4e05-abc2-ac27e0293f60"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:35.506885 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.506825 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d109fe04-a8c7-4e05-abc2-ac27e0293f60" (UID: "d109fe04-a8c7-4e05-abc2-ac27e0293f60"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:50:35.506963 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.506892 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-token" (OuterVolumeSpecName: "istio-token") pod "d109fe04-a8c7-4e05-abc2-ac27e0293f60" (UID: "d109fe04-a8c7-4e05-abc2-ac27e0293f60"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:50:35.506963 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.506914 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d109fe04-a8c7-4e05-abc2-ac27e0293f60-kube-api-access-qsqcs" (OuterVolumeSpecName: "kube-api-access-qsqcs") pod "d109fe04-a8c7-4e05-abc2-ac27e0293f60" (UID: "d109fe04-a8c7-4e05-abc2-ac27e0293f60"). InnerVolumeSpecName "kube-api-access-qsqcs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:50:35.605899 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.605801 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-data\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:35.605899 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.605833 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-envoy\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:35.605899 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.605841 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-podinfo\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:35.605899 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.605850 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d109fe04-a8c7-4e05-abc2-ac27e0293f60-istio-token\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:35.605899 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.605861 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-workload-certs\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:35.605899 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.605869 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-credential-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:35.605899 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.605879 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d109fe04-a8c7-4e05-abc2-ac27e0293f60-workload-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:35.605899 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.605888 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsqcs\" (UniqueName: \"kubernetes.io/projected/d109fe04-a8c7-4e05-abc2-ac27e0293f60-kube-api-access-qsqcs\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:35.812238 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.812204 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:50:35.816292 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.816259 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954"] Jul 12 18:50:35.822558 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:35.822522 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-73e47abc-openshift-default-8ddc8c8c5-jd954"] Jul 12 18:50:37.304581 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:37.304547 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d109fe04-a8c7-4e05-abc2-ac27e0293f60" path="/var/lib/kubelet/pods/d109fe04-a8c7-4e05-abc2-ac27e0293f60/volumes" Jul 12 18:50:41.294047 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.294007 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9"] Jul 12 18:50:41.294561 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.294545 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d109fe04-a8c7-4e05-abc2-ac27e0293f60" containerName="istio-proxy" Jul 12 18:50:41.294606 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.294564 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d109fe04-a8c7-4e05-abc2-ac27e0293f60" containerName="istio-proxy" Jul 12 18:50:41.294681 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.294670 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d109fe04-a8c7-4e05-abc2-ac27e0293f60" containerName="istio-proxy" Jul 12 18:50:41.298039 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.297979 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.300397 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.300375 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-1ad2a327-openshift-default-dockercfg-whxt6\"" Jul 12 18:50:41.308653 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.308626 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9"] Jul 12 18:50:41.356691 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.356652 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghx8j\" (UniqueName: \"kubernetes.io/projected/6f621f96-444b-4cf6-b06f-a32d79c421aa-kube-api-access-ghx8j\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.356691 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.356692 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-workload-socket\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.356899 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.356710 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f621f96-444b-4cf6-b06f-a32d79c421aa-istiod-ca-cert\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.356899 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.356777 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-token\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.356899 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.356818 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-credential-socket\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.356899 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.356853 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-envoy\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.356899 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.356873 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-podinfo\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.357122 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.356910 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-workload-certs\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.357122 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.356995 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-data\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.458464 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.458423 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-envoy\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.458464 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.458467 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-podinfo\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.458715 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.458490 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-workload-certs\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.458715 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.458533 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-data\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.458715 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.458616 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghx8j\" (UniqueName: \"kubernetes.io/projected/6f621f96-444b-4cf6-b06f-a32d79c421aa-kube-api-access-ghx8j\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.458715 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.458642 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-workload-socket\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.458715 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.458663 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f621f96-444b-4cf6-b06f-a32d79c421aa-istiod-ca-cert\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.458715 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.458692 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-token\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.459029 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.458719 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-credential-socket\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.459095 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.459029 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-workload-certs\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.459095 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.459029 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-workload-socket\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.459204 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.459159 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-data\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.459204 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.459191 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-credential-socket\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.459830 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.459805 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f621f96-444b-4cf6-b06f-a32d79c421aa-istiod-ca-cert\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.460855 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.460837 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-envoy\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.460987 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.460970 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-podinfo\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.472875 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.472847 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghx8j\" (UniqueName: \"kubernetes.io/projected/6f621f96-444b-4cf6-b06f-a32d79c421aa-kube-api-access-ghx8j\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.479663 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.479637 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-token\") pod \"e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.611480 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.611385 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:41.755168 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.755142 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9"] Jul 12 18:50:41.757157 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:50:41.757129 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f621f96_444b_4cf6_b06f_a32d79c421aa.slice/crio-72f5ca671745620288141f7dcea1b25fcccd25b713559dc935e1e604be9fa11c WatchSource:0}: Error finding container 72f5ca671745620288141f7dcea1b25fcccd25b713559dc935e1e604be9fa11c: Status 404 returned error can't find the container with id 72f5ca671745620288141f7dcea1b25fcccd25b713559dc935e1e604be9fa11c Jul 12 18:50:41.759257 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.759213 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:50:41.759365 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.759311 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:50:41.759430 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:41.759362 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:50:42.537778 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:42.537733 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" event={"ID":"6f621f96-444b-4cf6-b06f-a32d79c421aa","Type":"ContainerStarted","Data":"f309c911c95c35e1331e78f5e87083825a3bedaa3ffb8811bd958c3e507080ad"} Jul 12 18:50:42.538208 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:42.537802 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" event={"ID":"6f621f96-444b-4cf6-b06f-a32d79c421aa","Type":"ContainerStarted","Data":"72f5ca671745620288141f7dcea1b25fcccd25b713559dc935e1e604be9fa11c"} Jul 12 18:50:42.560003 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:42.559925 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" podStartSLOduration=1.559905594 podStartE2EDuration="1.559905594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:50:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:50:42.55719983 +0000 UTC m=+1533.756666862" watchObservedRunningTime="2026-07-12 18:50:42.559905594 +0000 UTC m=+1533.759372625" Jul 12 18:50:42.612552 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:42.612520 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:42.618511 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:42.618481 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:43.544463 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:43.544431 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:43.545638 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:43.545617 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:47.012992 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:47.012953 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9"] Jul 12 18:50:47.013436 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:47.013133 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" podUID="6f621f96-444b-4cf6-b06f-a32d79c421aa" containerName="istio-proxy" containerID="cri-o://f309c911c95c35e1331e78f5e87083825a3bedaa3ffb8811bd958c3e507080ad" gracePeriod=30 Jul 12 18:50:52.272597 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.272563 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:52.367214 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367102 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-workload-socket\") pod \"6f621f96-444b-4cf6-b06f-a32d79c421aa\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " Jul 12 18:50:52.367214 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367161 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghx8j\" (UniqueName: \"kubernetes.io/projected/6f621f96-444b-4cf6-b06f-a32d79c421aa-kube-api-access-ghx8j\") pod \"6f621f96-444b-4cf6-b06f-a32d79c421aa\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " Jul 12 18:50:52.367214 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367196 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f621f96-444b-4cf6-b06f-a32d79c421aa-istiod-ca-cert\") pod \"6f621f96-444b-4cf6-b06f-a32d79c421aa\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " Jul 12 18:50:52.367214 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367219 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-envoy\") pod \"6f621f96-444b-4cf6-b06f-a32d79c421aa\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " Jul 12 18:50:52.367574 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367237 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-podinfo\") pod \"6f621f96-444b-4cf6-b06f-a32d79c421aa\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " Jul 12 18:50:52.367574 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367322 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-workload-certs\") pod \"6f621f96-444b-4cf6-b06f-a32d79c421aa\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " Jul 12 18:50:52.367574 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367355 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-data\") pod \"6f621f96-444b-4cf6-b06f-a32d79c421aa\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " Jul 12 18:50:52.367574 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367403 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-token\") pod \"6f621f96-444b-4cf6-b06f-a32d79c421aa\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " Jul 12 18:50:52.367574 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367425 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-credential-socket\") pod \"6f621f96-444b-4cf6-b06f-a32d79c421aa\" (UID: \"6f621f96-444b-4cf6-b06f-a32d79c421aa\") " Jul 12 18:50:52.367574 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367466 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6f621f96-444b-4cf6-b06f-a32d79c421aa" (UID: "6f621f96-444b-4cf6-b06f-a32d79c421aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:52.367887 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367687 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "6f621f96-444b-4cf6-b06f-a32d79c421aa" (UID: "6f621f96-444b-4cf6-b06f-a32d79c421aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:52.367887 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367701 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f621f96-444b-4cf6-b06f-a32d79c421aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6f621f96-444b-4cf6-b06f-a32d79c421aa" (UID: "6f621f96-444b-4cf6-b06f-a32d79c421aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:50:52.367887 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367743 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-workload-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:52.367887 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367829 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6f621f96-444b-4cf6-b06f-a32d79c421aa" (UID: "6f621f96-444b-4cf6-b06f-a32d79c421aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:52.368125 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.367965 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6f621f96-444b-4cf6-b06f-a32d79c421aa" (UID: "6f621f96-444b-4cf6-b06f-a32d79c421aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:52.369683 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.369656 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6f621f96-444b-4cf6-b06f-a32d79c421aa" (UID: "6f621f96-444b-4cf6-b06f-a32d79c421aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:50:52.369807 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.369659 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "6f621f96-444b-4cf6-b06f-a32d79c421aa" (UID: "6f621f96-444b-4cf6-b06f-a32d79c421aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:50:52.369980 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.369955 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6f621f96-444b-4cf6-b06f-a32d79c421aa" (UID: "6f621f96-444b-4cf6-b06f-a32d79c421aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:52.369980 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.369972 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f621f96-444b-4cf6-b06f-a32d79c421aa-kube-api-access-ghx8j" (OuterVolumeSpecName: "kube-api-access-ghx8j") pod "6f621f96-444b-4cf6-b06f-a32d79c421aa" (UID: "6f621f96-444b-4cf6-b06f-a32d79c421aa"). InnerVolumeSpecName "kube-api-access-ghx8j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:50:52.469241 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.469202 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-token\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:52.469241 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.469242 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-credential-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:52.469462 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.469253 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghx8j\" (UniqueName: \"kubernetes.io/projected/6f621f96-444b-4cf6-b06f-a32d79c421aa-kube-api-access-ghx8j\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:52.469462 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.469262 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f621f96-444b-4cf6-b06f-a32d79c421aa-istiod-ca-cert\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:52.469462 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.469270 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-envoy\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:52.469462 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.469278 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-podinfo\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:52.469462 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.469285 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-workload-certs\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:52.469462 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.469292 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f621f96-444b-4cf6-b06f-a32d79c421aa-istio-data\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:50:52.595673 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.595638 2585 generic.go:356] "Generic (PLEG): container finished" podID="6f621f96-444b-4cf6-b06f-a32d79c421aa" containerID="f309c911c95c35e1331e78f5e87083825a3bedaa3ffb8811bd958c3e507080ad" exitCode=0 Jul 12 18:50:52.595856 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.595688 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" Jul 12 18:50:52.595856 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.595728 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" event={"ID":"6f621f96-444b-4cf6-b06f-a32d79c421aa","Type":"ContainerDied","Data":"f309c911c95c35e1331e78f5e87083825a3bedaa3ffb8811bd958c3e507080ad"} Jul 12 18:50:52.595856 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.595788 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9" event={"ID":"6f621f96-444b-4cf6-b06f-a32d79c421aa","Type":"ContainerDied","Data":"72f5ca671745620288141f7dcea1b25fcccd25b713559dc935e1e604be9fa11c"} Jul 12 18:50:52.595856 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.595813 2585 scope.go:117] "RemoveContainer" containerID="f309c911c95c35e1331e78f5e87083825a3bedaa3ffb8811bd958c3e507080ad" Jul 12 18:50:52.606360 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.606336 2585 scope.go:117] "RemoveContainer" containerID="f309c911c95c35e1331e78f5e87083825a3bedaa3ffb8811bd958c3e507080ad" Jul 12 18:50:52.606675 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:50:52.606656 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f309c911c95c35e1331e78f5e87083825a3bedaa3ffb8811bd958c3e507080ad\": container with ID starting with f309c911c95c35e1331e78f5e87083825a3bedaa3ffb8811bd958c3e507080ad not found: ID does not exist" containerID="f309c911c95c35e1331e78f5e87083825a3bedaa3ffb8811bd958c3e507080ad" Jul 12 18:50:52.606725 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.606686 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f309c911c95c35e1331e78f5e87083825a3bedaa3ffb8811bd958c3e507080ad"} err="failed to get container status \"f309c911c95c35e1331e78f5e87083825a3bedaa3ffb8811bd958c3e507080ad\": rpc error: code = NotFound desc = could not find container \"f309c911c95c35e1331e78f5e87083825a3bedaa3ffb8811bd958c3e507080ad\": container with ID starting with f309c911c95c35e1331e78f5e87083825a3bedaa3ffb8811bd958c3e507080ad not found: ID does not exist" Jul 12 18:50:52.618472 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.618378 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9"] Jul 12 18:50:52.622051 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:52.622019 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-1ad2a327-openshift-default-6ffd54d7b7-pm5x9"] Jul 12 18:50:53.305905 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:50:53.305867 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f621f96-444b-4cf6-b06f-a32d79c421aa" path="/var/lib/kubelet/pods/6f621f96-444b-4cf6-b06f-a32d79c421aa/volumes" Jul 12 18:52:17.905143 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:17.905050 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:52:27.893709 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:27.893672 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc"] Jul 12 18:52:27.894118 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:27.894096 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f621f96-444b-4cf6-b06f-a32d79c421aa" containerName="istio-proxy" Jul 12 18:52:27.894118 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:27.894108 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f621f96-444b-4cf6-b06f-a32d79c421aa" containerName="istio-proxy" Jul 12 18:52:27.894197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:27.894180 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f621f96-444b-4cf6-b06f-a32d79c421aa" containerName="istio-proxy" Jul 12 18:52:27.897313 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:27.897295 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:27.899768 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:27.899746 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1784bded-openshift-default-dockercfg-447kc\"" Jul 12 18:52:27.916526 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:27.916497 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc"] Jul 12 18:52:28.046153 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.046113 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-workload-socket\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.046153 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.046157 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-envoy\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.046402 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.046173 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-data\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.046402 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.046189 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6gf9\" (UniqueName: \"kubernetes.io/projected/6d29993f-97bc-4cdd-9c08-199214aeec26-kube-api-access-c6gf9\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.046402 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.046237 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-token\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.046402 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.046282 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-credential-socket\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.046402 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.046296 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-podinfo\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.046402 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.046352 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d29993f-97bc-4cdd-9c08-199214aeec26-istiod-ca-cert\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.046402 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.046399 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-workload-certs\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.147127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.147019 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-credential-socket\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.147127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.147061 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-podinfo\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.147127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.147093 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d29993f-97bc-4cdd-9c08-199214aeec26-istiod-ca-cert\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.147127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.147120 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-workload-certs\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.147474 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.147151 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-workload-socket\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.147474 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.147171 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-envoy\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.147474 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.147219 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-data\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.147474 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.147262 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6gf9\" (UniqueName: \"kubernetes.io/projected/6d29993f-97bc-4cdd-9c08-199214aeec26-kube-api-access-c6gf9\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.147474 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.147307 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-token\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.147730 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.147529 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-credential-socket\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.147788 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.147723 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-data\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.147788 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.147766 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-workload-socket\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.147860 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.147803 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-workload-certs\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.148084 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.148064 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d29993f-97bc-4cdd-9c08-199214aeec26-istiod-ca-cert\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.149533 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.149514 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-envoy\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.149791 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.149772 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-podinfo\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.155247 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.155226 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-token\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.155391 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.155372 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6gf9\" (UniqueName: \"kubernetes.io/projected/6d29993f-97bc-4cdd-9c08-199214aeec26-kube-api-access-c6gf9\") pod \"e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.208151 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.208113 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:28.353970 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.353925 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc"] Jul 12 18:52:28.355402 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:52:28.355374 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d29993f_97bc_4cdd_9c08_199214aeec26.slice/crio-25595ee21fa7452ea5b8f9338e92f83b8c735cdf6299558807792c6a20aaa984 WatchSource:0}: Error finding container 25595ee21fa7452ea5b8f9338e92f83b8c735cdf6299558807792c6a20aaa984: Status 404 returned error can't find the container with id 25595ee21fa7452ea5b8f9338e92f83b8c735cdf6299558807792c6a20aaa984 Jul 12 18:52:28.357507 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.357472 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:52:28.357609 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.357540 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:52:28.357609 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:28.357572 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:52:29.115943 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:29.115889 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" event={"ID":"6d29993f-97bc-4cdd-9c08-199214aeec26","Type":"ContainerStarted","Data":"4eb31396006ec50a8ecb87618b706d2235fef3a981890fd272c894616ddb12fa"} Jul 12 18:52:29.116439 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:29.115964 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" event={"ID":"6d29993f-97bc-4cdd-9c08-199214aeec26","Type":"ContainerStarted","Data":"25595ee21fa7452ea5b8f9338e92f83b8c735cdf6299558807792c6a20aaa984"} Jul 12 18:52:29.137690 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:29.137637 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" podStartSLOduration=2.137618924 podStartE2EDuration="2.137618924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:52:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:52:29.135099724 +0000 UTC m=+1640.334566754" watchObservedRunningTime="2026-07-12 18:52:29.137618924 +0000 UTC m=+1640.337085955" Jul 12 18:52:29.209085 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:29.209043 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:29.210697 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:29.210672 2585 patch_prober.go:28] interesting pod/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 12 18:52:29.210778 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:29.210723 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" podUID="6d29993f-97bc-4cdd-9c08-199214aeec26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 12 18:52:30.212022 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:30.211985 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:52:30.213839 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:30.213819 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:31.127506 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:31.127455 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:31.128748 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:31.128725 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:41.611875 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:41.611834 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:52:46.129772 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:46.129668 2585 patch_prober.go:28] interesting pod/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 12 18:52:46.129772 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:46.129750 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" podUID="6d29993f-97bc-4cdd-9c08-199214aeec26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 12 18:52:46.216760 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:46.216725 2585 generic.go:356] "Generic (PLEG): container finished" podID="6d29993f-97bc-4cdd-9c08-199214aeec26" containerID="4eb31396006ec50a8ecb87618b706d2235fef3a981890fd272c894616ddb12fa" exitCode=0 Jul 12 18:52:46.216953 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:46.216795 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" event={"ID":"6d29993f-97bc-4cdd-9c08-199214aeec26","Type":"ContainerDied","Data":"4eb31396006ec50a8ecb87618b706d2235fef3a981890fd272c894616ddb12fa"} Jul 12 18:52:46.217227 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:46.217211 2585 scope.go:117] "RemoveContainer" containerID="4eb31396006ec50a8ecb87618b706d2235fef3a981890fd272c894616ddb12fa" Jul 12 18:52:46.218278 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:46.218247 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:52:46.218359 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:46.218338 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:52:46.218397 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:46.218385 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:52:47.224809 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:47.224765 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" event={"ID":"6d29993f-97bc-4cdd-9c08-199214aeec26","Type":"ContainerStarted","Data":"5bc32ac08f1a438eb9a0cbafd14f2897d5bfc1df9200c48e5ab0167082528d3a"} Jul 12 18:52:48.208641 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:48.208596 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:48.212713 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:48.212680 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:48.232152 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:48.232123 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:48.233216 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:48.233195 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:48.820943 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:48.820894 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:52:48.970100 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:48.970066 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc"] Jul 12 18:52:49.237121 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:49.237019 2585 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" secret="" err="secret \"e2e-mt-1784bded-openshift-default-dockercfg-447kc\" not found" Jul 12 18:52:50.242682 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:50.242642 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" podUID="6d29993f-97bc-4cdd-9c08-199214aeec26" containerName="istio-proxy" containerID="cri-o://5bc32ac08f1a438eb9a0cbafd14f2897d5bfc1df9200c48e5ab0167082528d3a" gracePeriod=30 Jul 12 18:52:55.508582 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.508511 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:55.611503 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.611460 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d29993f-97bc-4cdd-9c08-199214aeec26-istiod-ca-cert\") pod \"6d29993f-97bc-4cdd-9c08-199214aeec26\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " Jul 12 18:52:55.611708 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.611526 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-workload-socket\") pod \"6d29993f-97bc-4cdd-9c08-199214aeec26\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " Jul 12 18:52:55.611708 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.611575 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-data\") pod \"6d29993f-97bc-4cdd-9c08-199214aeec26\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " Jul 12 18:52:55.611708 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.611605 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-workload-certs\") pod \"6d29993f-97bc-4cdd-9c08-199214aeec26\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " Jul 12 18:52:55.611708 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.611633 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-credential-socket\") pod \"6d29993f-97bc-4cdd-9c08-199214aeec26\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " Jul 12 18:52:55.611708 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.611670 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6gf9\" (UniqueName: \"kubernetes.io/projected/6d29993f-97bc-4cdd-9c08-199214aeec26-kube-api-access-c6gf9\") pod \"6d29993f-97bc-4cdd-9c08-199214aeec26\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " Jul 12 18:52:55.611708 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.611696 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-podinfo\") pod \"6d29993f-97bc-4cdd-9c08-199214aeec26\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " Jul 12 18:52:55.612065 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.611733 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-envoy\") pod \"6d29993f-97bc-4cdd-9c08-199214aeec26\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " Jul 12 18:52:55.612065 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.611759 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-token\") pod \"6d29993f-97bc-4cdd-9c08-199214aeec26\" (UID: \"6d29993f-97bc-4cdd-9c08-199214aeec26\") " Jul 12 18:52:55.612065 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.611966 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d29993f-97bc-4cdd-9c08-199214aeec26" (UID: "6d29993f-97bc-4cdd-9c08-199214aeec26"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:52:55.612065 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.611988 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d29993f-97bc-4cdd-9c08-199214aeec26" (UID: "6d29993f-97bc-4cdd-9c08-199214aeec26"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:52:55.612065 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.612005 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d29993f-97bc-4cdd-9c08-199214aeec26-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d29993f-97bc-4cdd-9c08-199214aeec26" (UID: "6d29993f-97bc-4cdd-9c08-199214aeec26"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:52:55.612330 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.611995 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d29993f-97bc-4cdd-9c08-199214aeec26" (UID: "6d29993f-97bc-4cdd-9c08-199214aeec26"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:52:55.612330 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.612168 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-workload-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:52:55.612330 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.612189 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-workload-certs\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:52:55.612330 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.612204 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-credential-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:52:55.612330 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.612219 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d29993f-97bc-4cdd-9c08-199214aeec26-istiod-ca-cert\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:52:55.612724 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.612696 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d29993f-97bc-4cdd-9c08-199214aeec26" (UID: "6d29993f-97bc-4cdd-9c08-199214aeec26"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:52:55.614129 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.614097 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d29993f-97bc-4cdd-9c08-199214aeec26" (UID: "6d29993f-97bc-4cdd-9c08-199214aeec26"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:52:55.614239 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.614191 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d29993f-97bc-4cdd-9c08-199214aeec26" (UID: "6d29993f-97bc-4cdd-9c08-199214aeec26"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:52:55.614395 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.614367 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d29993f-97bc-4cdd-9c08-199214aeec26" (UID: "6d29993f-97bc-4cdd-9c08-199214aeec26"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:52:55.614437 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.614371 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d29993f-97bc-4cdd-9c08-199214aeec26-kube-api-access-c6gf9" (OuterVolumeSpecName: "kube-api-access-c6gf9") pod "6d29993f-97bc-4cdd-9c08-199214aeec26" (UID: "6d29993f-97bc-4cdd-9c08-199214aeec26"). InnerVolumeSpecName "kube-api-access-c6gf9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:52:55.712714 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.712675 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-envoy\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:52:55.712714 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.712707 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-token\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:52:55.712714 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.712717 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-data\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:52:55.712714 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.712726 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6gf9\" (UniqueName: \"kubernetes.io/projected/6d29993f-97bc-4cdd-9c08-199214aeec26-kube-api-access-c6gf9\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:52:55.713044 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:55.712736 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d29993f-97bc-4cdd-9c08-199214aeec26-istio-podinfo\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:52:56.276605 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:56.276568 2585 generic.go:356] "Generic (PLEG): container finished" podID="6d29993f-97bc-4cdd-9c08-199214aeec26" containerID="5bc32ac08f1a438eb9a0cbafd14f2897d5bfc1df9200c48e5ab0167082528d3a" exitCode=0 Jul 12 18:52:56.276802 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:56.276620 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" Jul 12 18:52:56.276802 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:56.276644 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" event={"ID":"6d29993f-97bc-4cdd-9c08-199214aeec26","Type":"ContainerDied","Data":"5bc32ac08f1a438eb9a0cbafd14f2897d5bfc1df9200c48e5ab0167082528d3a"} Jul 12 18:52:56.276802 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:56.276696 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc" event={"ID":"6d29993f-97bc-4cdd-9c08-199214aeec26","Type":"ContainerDied","Data":"25595ee21fa7452ea5b8f9338e92f83b8c735cdf6299558807792c6a20aaa984"} Jul 12 18:52:56.276802 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:56.276720 2585 scope.go:117] "RemoveContainer" containerID="5bc32ac08f1a438eb9a0cbafd14f2897d5bfc1df9200c48e5ab0167082528d3a" Jul 12 18:52:56.289102 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:56.289072 2585 scope.go:117] "RemoveContainer" containerID="4eb31396006ec50a8ecb87618b706d2235fef3a981890fd272c894616ddb12fa" Jul 12 18:52:56.299388 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:56.299365 2585 scope.go:117] "RemoveContainer" containerID="5bc32ac08f1a438eb9a0cbafd14f2897d5bfc1df9200c48e5ab0167082528d3a" Jul 12 18:52:56.299740 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:52:56.299710 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bc32ac08f1a438eb9a0cbafd14f2897d5bfc1df9200c48e5ab0167082528d3a\": container with ID starting with 5bc32ac08f1a438eb9a0cbafd14f2897d5bfc1df9200c48e5ab0167082528d3a not found: ID does not exist" containerID="5bc32ac08f1a438eb9a0cbafd14f2897d5bfc1df9200c48e5ab0167082528d3a" Jul 12 18:52:56.299801 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:56.299752 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bc32ac08f1a438eb9a0cbafd14f2897d5bfc1df9200c48e5ab0167082528d3a"} err="failed to get container status \"5bc32ac08f1a438eb9a0cbafd14f2897d5bfc1df9200c48e5ab0167082528d3a\": rpc error: code = NotFound desc = could not find container \"5bc32ac08f1a438eb9a0cbafd14f2897d5bfc1df9200c48e5ab0167082528d3a\": container with ID starting with 5bc32ac08f1a438eb9a0cbafd14f2897d5bfc1df9200c48e5ab0167082528d3a not found: ID does not exist" Jul 12 18:52:56.299801 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:56.299770 2585 scope.go:117] "RemoveContainer" containerID="4eb31396006ec50a8ecb87618b706d2235fef3a981890fd272c894616ddb12fa" Jul 12 18:52:56.300017 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:52:56.299998 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4eb31396006ec50a8ecb87618b706d2235fef3a981890fd272c894616ddb12fa\": container with ID starting with 4eb31396006ec50a8ecb87618b706d2235fef3a981890fd272c894616ddb12fa not found: ID does not exist" containerID="4eb31396006ec50a8ecb87618b706d2235fef3a981890fd272c894616ddb12fa" Jul 12 18:52:56.300062 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:56.300021 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4eb31396006ec50a8ecb87618b706d2235fef3a981890fd272c894616ddb12fa"} err="failed to get container status \"4eb31396006ec50a8ecb87618b706d2235fef3a981890fd272c894616ddb12fa\": rpc error: code = NotFound desc = could not find container \"4eb31396006ec50a8ecb87618b706d2235fef3a981890fd272c894616ddb12fa\": container with ID starting with 4eb31396006ec50a8ecb87618b706d2235fef3a981890fd272c894616ddb12fa not found: ID does not exist" Jul 12 18:52:56.301126 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:56.301106 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc"] Jul 12 18:52:56.305097 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:56.305069 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1784bded-openshift-default-6cb5ff66f5-fffqc"] Jul 12 18:52:57.304947 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:52:57.304891 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d29993f-97bc-4cdd-9c08-199214aeec26" path="/var/lib/kubelet/pods/6d29993f-97bc-4cdd-9c08-199214aeec26/volumes" Jul 12 18:53:08.171073 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.171037 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl"] Jul 12 18:53:08.171634 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.171576 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d29993f-97bc-4cdd-9c08-199214aeec26" containerName="istio-proxy" Jul 12 18:53:08.171634 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.171596 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d29993f-97bc-4cdd-9c08-199214aeec26" containerName="istio-proxy" Jul 12 18:53:08.171753 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.171701 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d29993f-97bc-4cdd-9c08-199214aeec26" containerName="istio-proxy" Jul 12 18:53:08.176416 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.176393 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.178642 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.178621 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-443e5c6f-openshift-default-dockercfg-vchbd\"" Jul 12 18:53:08.186774 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.186747 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl"] Jul 12 18:53:08.325811 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.325769 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-podinfo\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.326025 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.325840 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-credential-socket\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.326025 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.325862 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-workload-certs\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.326025 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.325882 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-token\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.326025 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.325955 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-data\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.326025 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.325995 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istiod-ca-cert\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.326025 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.326016 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt6c7\" (UniqueName: \"kubernetes.io/projected/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-kube-api-access-xt6c7\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.326232 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.326041 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-workload-socket\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.326232 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.326064 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-envoy\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.427012 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.426901 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-data\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.427012 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.427004 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istiod-ca-cert\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.427251 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.427032 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt6c7\" (UniqueName: \"kubernetes.io/projected/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-kube-api-access-xt6c7\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.427251 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.427064 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-workload-socket\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.427251 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.427095 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-envoy\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.427251 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.427149 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-podinfo\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.427251 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.427221 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-credential-socket\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.427251 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.427251 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-workload-certs\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.427592 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.427273 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-token\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.427592 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.427313 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-data\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.427592 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.427392 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-workload-socket\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.427592 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.427571 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-credential-socket\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.427765 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.427625 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-workload-certs\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.427856 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.427836 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istiod-ca-cert\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.429383 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.429354 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-envoy\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.429667 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.429648 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-podinfo\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.435238 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.435203 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt6c7\" (UniqueName: \"kubernetes.io/projected/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-kube-api-access-xt6c7\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.435582 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.435559 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-token\") pod \"e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.487591 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.487555 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:08.633903 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.633878 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl"] Jul 12 18:53:08.635525 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:53:08.635485 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b13b46d_b39f_4166_a3b2_b0c8f4e35be3.slice/crio-02b305d0877aaefe88fcac12806d15dc002904ff45bce1c952a1c1a521d8f85c WatchSource:0}: Error finding container 02b305d0877aaefe88fcac12806d15dc002904ff45bce1c952a1c1a521d8f85c: Status 404 returned error can't find the container with id 02b305d0877aaefe88fcac12806d15dc002904ff45bce1c952a1c1a521d8f85c Jul 12 18:53:08.637757 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.637706 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:53:08.637846 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.637790 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:53:08.637846 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:08.637819 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:53:09.349815 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:09.349773 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" event={"ID":"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3","Type":"ContainerStarted","Data":"3176684bccb5d7cdade1546d73bcb3eb8db680a2b7d28ecc324221faa6be3b49"} Jul 12 18:53:09.350233 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:09.349835 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" event={"ID":"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3","Type":"ContainerStarted","Data":"02b305d0877aaefe88fcac12806d15dc002904ff45bce1c952a1c1a521d8f85c"} Jul 12 18:53:09.372039 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:09.371978 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" podStartSLOduration=1.371953527 podStartE2EDuration="1.371953527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:53:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:53:09.368270539 +0000 UTC m=+1680.567737567" watchObservedRunningTime="2026-07-12 18:53:09.371953527 +0000 UTC m=+1680.571420558" Jul 12 18:53:09.488082 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:09.487974 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:09.490575 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:09.490168 2585 patch_prober.go:28] interesting pod/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 12 18:53:09.490575 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:09.490253 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" podUID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 12 18:53:10.492665 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:10.492634 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:11.360291 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:11.360260 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:11.361287 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:11.361267 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:17.110580 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:17.110526 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:53:25.443171 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:25.443061 2585 generic.go:356] "Generic (PLEG): container finished" podID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerID="3176684bccb5d7cdade1546d73bcb3eb8db680a2b7d28ecc324221faa6be3b49" exitCode=0 Jul 12 18:53:25.443171 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:25.443133 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" event={"ID":"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3","Type":"ContainerDied","Data":"3176684bccb5d7cdade1546d73bcb3eb8db680a2b7d28ecc324221faa6be3b49"} Jul 12 18:53:25.443608 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:25.443535 2585 scope.go:117] "RemoveContainer" containerID="3176684bccb5d7cdade1546d73bcb3eb8db680a2b7d28ecc324221faa6be3b49" Jul 12 18:53:25.444555 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:25.444525 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:53:25.444623 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:25.444599 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:53:25.444666 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:25.444633 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:53:26.360949 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:26.360889 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:26.452614 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:26.452574 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" event={"ID":"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3","Type":"ContainerStarted","Data":"77e65e62e4f37e7b9c5da5565a89415670d37e477e67ca906865471138781f16"} Jul 12 18:53:26.488257 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:26.488215 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:26.489536 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:26.489510 2585 patch_prober.go:28] interesting pod/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 12 18:53:26.489660 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:26.489562 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" podUID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 12 18:53:27.488267 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:27.488226 2585 patch_prober.go:28] interesting pod/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 12 18:53:27.488708 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:27.488290 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" podUID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 12 18:53:28.488293 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:28.488258 2585 patch_prober.go:28] interesting pod/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 12 18:53:28.488726 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:28.488313 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" podUID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 12 18:53:29.488650 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:29.488618 2585 patch_prober.go:28] interesting pod/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 12 18:53:29.489150 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:29.488680 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" podUID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 12 18:53:30.491879 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:30.491845 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:30.492370 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:30.492126 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:30.492905 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:30.492888 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:53:42.307066 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:42.306978 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:53:55.815946 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:53:55.815902 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:54:06.115978 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:54:06.115922 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:54:16.409998 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:54:16.409960 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:54:48.713455 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:54:48.713413 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:54:58.287877 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:54:58.287836 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl"] Jul 12 18:54:58.288481 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:54:58.288059 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" podUID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerName="istio-proxy" containerID="cri-o://77e65e62e4f37e7b9c5da5565a89415670d37e477e67ca906865471138781f16" gracePeriod=30 Jul 12 18:55:00.494778 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:00.494732 2585 patch_prober.go:28] interesting pod/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 12 18:55:00.495190 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:00.494803 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" podUID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 18:55:03.533061 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.533034 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:55:03.601855 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.601744 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-envoy\") pod \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " Jul 12 18:55:03.601855 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.601791 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-workload-certs\") pod \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " Jul 12 18:55:03.601855 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.601841 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xt6c7\" (UniqueName: \"kubernetes.io/projected/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-kube-api-access-xt6c7\") pod \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " Jul 12 18:55:03.602174 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.601879 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-podinfo\") pod \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " Jul 12 18:55:03.602174 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.601906 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istiod-ca-cert\") pod \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " Jul 12 18:55:03.602174 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.601979 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-workload-socket\") pod \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " Jul 12 18:55:03.602174 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.602010 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-credential-socket\") pod \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " Jul 12 18:55:03.602174 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.602155 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-data\") pod \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " Jul 12 18:55:03.602418 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.602206 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" (UID: "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:55:03.602418 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.602235 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-token\") pod \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\" (UID: \"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3\") " Jul 12 18:55:03.602527 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.602490 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" (UID: "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:55:03.602581 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.602537 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" (UID: "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:55:03.602581 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.602555 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-workload-certs\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:55:03.602581 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.602574 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-workload-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:55:03.602734 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.602592 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" (UID: "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:55:03.603185 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.603162 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-data" (OuterVolumeSpecName: "istio-data") pod "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" (UID: "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:55:03.604254 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.604222 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" (UID: "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:55:03.604376 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.604356 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-token" (OuterVolumeSpecName: "istio-token") pod "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" (UID: "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:55:03.604470 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.604453 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" (UID: "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:55:03.604586 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.604571 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-kube-api-access-xt6c7" (OuterVolumeSpecName: "kube-api-access-xt6c7") pod "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" (UID: "7b13b46d-b39f-4166-a3b2-b0c8f4e35be3"). InnerVolumeSpecName "kube-api-access-xt6c7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:55:03.704066 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.704026 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-data\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:55:03.704066 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.704059 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-token\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:55:03.704066 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.704068 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-envoy\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:55:03.704066 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.704076 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xt6c7\" (UniqueName: \"kubernetes.io/projected/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-kube-api-access-xt6c7\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:55:03.704345 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.704087 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istio-podinfo\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:55:03.704345 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.704095 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-istiod-ca-cert\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:55:03.704345 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.704104 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3-credential-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:55:03.994766 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.994670 2585 generic.go:356] "Generic (PLEG): container finished" podID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerID="77e65e62e4f37e7b9c5da5565a89415670d37e477e67ca906865471138781f16" exitCode=0 Jul 12 18:55:03.994766 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.994721 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" Jul 12 18:55:03.995006 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.994755 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" event={"ID":"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3","Type":"ContainerDied","Data":"77e65e62e4f37e7b9c5da5565a89415670d37e477e67ca906865471138781f16"} Jul 12 18:55:03.995006 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.994809 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl" event={"ID":"7b13b46d-b39f-4166-a3b2-b0c8f4e35be3","Type":"ContainerDied","Data":"02b305d0877aaefe88fcac12806d15dc002904ff45bce1c952a1c1a521d8f85c"} Jul 12 18:55:03.995006 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:03.994834 2585 scope.go:117] "RemoveContainer" containerID="77e65e62e4f37e7b9c5da5565a89415670d37e477e67ca906865471138781f16" Jul 12 18:55:04.004743 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:04.004725 2585 scope.go:117] "RemoveContainer" containerID="3176684bccb5d7cdade1546d73bcb3eb8db680a2b7d28ecc324221faa6be3b49" Jul 12 18:55:04.013440 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:04.013414 2585 scope.go:117] "RemoveContainer" containerID="77e65e62e4f37e7b9c5da5565a89415670d37e477e67ca906865471138781f16" Jul 12 18:55:04.013786 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:55:04.013763 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77e65e62e4f37e7b9c5da5565a89415670d37e477e67ca906865471138781f16\": container with ID starting with 77e65e62e4f37e7b9c5da5565a89415670d37e477e67ca906865471138781f16 not found: ID does not exist" containerID="77e65e62e4f37e7b9c5da5565a89415670d37e477e67ca906865471138781f16" Jul 12 18:55:04.013832 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:04.013797 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77e65e62e4f37e7b9c5da5565a89415670d37e477e67ca906865471138781f16"} err="failed to get container status \"77e65e62e4f37e7b9c5da5565a89415670d37e477e67ca906865471138781f16\": rpc error: code = NotFound desc = could not find container \"77e65e62e4f37e7b9c5da5565a89415670d37e477e67ca906865471138781f16\": container with ID starting with 77e65e62e4f37e7b9c5da5565a89415670d37e477e67ca906865471138781f16 not found: ID does not exist" Jul 12 18:55:04.013832 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:04.013816 2585 scope.go:117] "RemoveContainer" containerID="3176684bccb5d7cdade1546d73bcb3eb8db680a2b7d28ecc324221faa6be3b49" Jul 12 18:55:04.014125 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:55:04.014102 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3176684bccb5d7cdade1546d73bcb3eb8db680a2b7d28ecc324221faa6be3b49\": container with ID starting with 3176684bccb5d7cdade1546d73bcb3eb8db680a2b7d28ecc324221faa6be3b49 not found: ID does not exist" containerID="3176684bccb5d7cdade1546d73bcb3eb8db680a2b7d28ecc324221faa6be3b49" Jul 12 18:55:04.014261 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:04.014130 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3176684bccb5d7cdade1546d73bcb3eb8db680a2b7d28ecc324221faa6be3b49"} err="failed to get container status \"3176684bccb5d7cdade1546d73bcb3eb8db680a2b7d28ecc324221faa6be3b49\": rpc error: code = NotFound desc = could not find container \"3176684bccb5d7cdade1546d73bcb3eb8db680a2b7d28ecc324221faa6be3b49\": container with ID starting with 3176684bccb5d7cdade1546d73bcb3eb8db680a2b7d28ecc324221faa6be3b49 not found: ID does not exist" Jul 12 18:55:04.017888 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:04.017857 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl"] Jul 12 18:55:04.023450 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:04.023420 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-443e5c6f-openshift-default-6fc99986cf-wjmwl"] Jul 12 18:55:04.313794 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:04.313752 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:55:05.304853 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:05.304814 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" path="/var/lib/kubelet/pods/7b13b46d-b39f-4166-a3b2-b0c8f4e35be3/volumes" Jul 12 18:55:09.434984 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:09.434957 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:55:09.439617 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:09.439589 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 18:55:16.221723 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:16.221662 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:55:52.979633 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:52.979591 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg"] Jul 12 18:55:52.980272 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:52.980155 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerName="istio-proxy" Jul 12 18:55:52.980272 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:52.980171 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerName="istio-proxy" Jul 12 18:55:52.980272 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:52.980187 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerName="istio-proxy" Jul 12 18:55:52.980272 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:52.980193 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerName="istio-proxy" Jul 12 18:55:52.980272 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:52.980204 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d29993f-97bc-4cdd-9c08-199214aeec26" containerName="istio-proxy" Jul 12 18:55:52.980272 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:52.980210 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d29993f-97bc-4cdd-9c08-199214aeec26" containerName="istio-proxy" Jul 12 18:55:52.980272 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:52.980272 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d29993f-97bc-4cdd-9c08-199214aeec26" containerName="istio-proxy" Jul 12 18:55:52.980501 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:52.980281 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerName="istio-proxy" Jul 12 18:55:52.983310 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:52.983294 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:52.985826 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:52.985801 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-58790d-openshift-default-dockercfg-tjr2f\"" Jul 12 18:55:52.995655 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:52.995630 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg"] Jul 12 18:55:53.052232 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.052198 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/574dea44-9ecf-4c1b-abba-526e442b081d-istiod-ca-cert\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.052232 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.052234 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/574dea44-9ecf-4c1b-abba-526e442b081d-istio-token\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.052453 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.052267 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/574dea44-9ecf-4c1b-abba-526e442b081d-istio-podinfo\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.052453 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.052296 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb4q8\" (UniqueName: \"kubernetes.io/projected/574dea44-9ecf-4c1b-abba-526e442b081d-kube-api-access-hb4q8\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.052453 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.052364 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-workload-socket\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.052453 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.052413 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-credential-socket\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.052453 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.052439 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-istio-data\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.052641 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.052477 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-istio-envoy\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.052641 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.052570 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-workload-certs\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.154101 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.154056 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/574dea44-9ecf-4c1b-abba-526e442b081d-istio-podinfo\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.154388 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.154358 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb4q8\" (UniqueName: \"kubernetes.io/projected/574dea44-9ecf-4c1b-abba-526e442b081d-kube-api-access-hb4q8\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.154519 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.154403 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-workload-socket\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.154519 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.154440 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-credential-socket\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.154519 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.154462 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-istio-data\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.154519 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.154502 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-istio-envoy\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.154741 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.154585 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-workload-certs\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.154741 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.154673 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/574dea44-9ecf-4c1b-abba-526e442b081d-istiod-ca-cert\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.154741 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.154702 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/574dea44-9ecf-4c1b-abba-526e442b081d-istio-token\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.155000 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.154826 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-workload-socket\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.155099 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.155079 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-credential-socket\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.155213 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.155124 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-workload-certs\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.155393 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.155368 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-istio-data\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.155707 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.155683 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/574dea44-9ecf-4c1b-abba-526e442b081d-istiod-ca-cert\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.157289 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.157269 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/574dea44-9ecf-4c1b-abba-526e442b081d-istio-podinfo\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.157410 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.157307 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-istio-envoy\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.168612 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.168574 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb4q8\" (UniqueName: \"kubernetes.io/projected/574dea44-9ecf-4c1b-abba-526e442b081d-kube-api-access-hb4q8\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.169283 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.169254 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/574dea44-9ecf-4c1b-abba-526e442b081d-istio-token\") pod \"e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.294510 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.294475 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:53.435195 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.435165 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg"] Jul 12 18:55:53.436865 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:55:53.436829 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod574dea44_9ecf_4c1b_abba_526e442b081d.slice/crio-b1857841b63428d577b190b139edf7e5694cf51285a9a3229417ceeab6265d0c WatchSource:0}: Error finding container b1857841b63428d577b190b139edf7e5694cf51285a9a3229417ceeab6265d0c: Status 404 returned error can't find the container with id b1857841b63428d577b190b139edf7e5694cf51285a9a3229417ceeab6265d0c Jul 12 18:55:53.439081 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.439038 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:55:53.439197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.439123 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:55:53.439197 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:53.439169 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:55:54.271310 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:54.271273 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" event={"ID":"574dea44-9ecf-4c1b-abba-526e442b081d","Type":"ContainerStarted","Data":"aba2ffd3c3142754db8861eb57176046008cc17131a4a4c5716ca70fd3721e98"} Jul 12 18:55:54.271706 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:54.271328 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" event={"ID":"574dea44-9ecf-4c1b-abba-526e442b081d","Type":"ContainerStarted","Data":"b1857841b63428d577b190b139edf7e5694cf51285a9a3229417ceeab6265d0c"} Jul 12 18:55:54.294727 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:54.294674 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" podStartSLOduration=2.294658309 podStartE2EDuration="2.294658309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:55:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:55:54.29146082 +0000 UTC m=+1845.490927849" watchObservedRunningTime="2026-07-12 18:55:54.294658309 +0000 UTC m=+1845.494125341" Jul 12 18:55:54.294907 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:54.294705 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:54.299802 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:54.299774 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:55.277085 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:55.277048 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:55.278118 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:55.278098 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:55:58.816114 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:55:58.816072 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:56:01.260619 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.260587 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6"] Jul 12 18:56:01.261238 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.261218 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b13b46d-b39f-4166-a3b2-b0c8f4e35be3" containerName="istio-proxy" Jul 12 18:56:01.265728 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.265694 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.268298 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.268273 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-47ad52-openshift-default-dockercfg-q6jwm\"" Jul 12 18:56:01.276981 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.276948 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6"] Jul 12 18:56:01.343956 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.343390 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-workload-certs\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.343956 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.343448 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-data\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.343956 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.343499 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-credential-socket\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.343956 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.343528 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9cmm\" (UniqueName: \"kubernetes.io/projected/31d9f1be-4d1e-4b02-926b-07829e2f58e4-kube-api-access-h9cmm\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.343956 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.343576 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-podinfo\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.343956 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.343607 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-envoy\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.343956 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.343658 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-workload-socket\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.343956 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.343729 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istiod-ca-cert\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.343956 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.343757 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-token\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.445011 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.444965 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istiod-ca-cert\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.445011 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.445023 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-token\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.445306 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.445087 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-workload-certs\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.445306 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.445114 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-data\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.445306 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.445150 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-credential-socket\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.445306 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.445174 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9cmm\" (UniqueName: \"kubernetes.io/projected/31d9f1be-4d1e-4b02-926b-07829e2f58e4-kube-api-access-h9cmm\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.445306 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.445219 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-podinfo\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.445306 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.445247 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-envoy\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.445599 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.445329 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-workload-socket\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.445782 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.445750 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-credential-socket\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.448819 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.448664 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istiod-ca-cert\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.448819 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.446638 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-workload-certs\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.448819 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.446376 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-workload-socket\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.449102 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.449048 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-data\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.452984 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.449373 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-envoy\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.457089 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.457005 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-podinfo\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.458854 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.458819 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9cmm\" (UniqueName: \"kubernetes.io/projected/31d9f1be-4d1e-4b02-926b-07829e2f58e4-kube-api-access-h9cmm\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.459343 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.459137 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-token\") pod \"e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.579842 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.579789 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:01.774031 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.773997 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6"] Jul 12 18:56:01.775904 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:56:01.775869 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31d9f1be_4d1e_4b02_926b_07829e2f58e4.slice/crio-718d2f030af23bf27e06a146162b928fb232462c21fb74fa481368feeec63641 WatchSource:0}: Error finding container 718d2f030af23bf27e06a146162b928fb232462c21fb74fa481368feeec63641: Status 404 returned error can't find the container with id 718d2f030af23bf27e06a146162b928fb232462c21fb74fa481368feeec63641 Jul 12 18:56:01.778150 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.778114 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:56:01.778251 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.778198 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:56:01.778313 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:01.778245 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:56:02.319037 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:02.318998 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" event={"ID":"31d9f1be-4d1e-4b02-926b-07829e2f58e4","Type":"ContainerStarted","Data":"1fa58b71d4298ff625ad90e114bdf7492d481d354cb7427bfbb20297c1df6dd9"} Jul 12 18:56:02.319467 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:02.319055 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" event={"ID":"31d9f1be-4d1e-4b02-926b-07829e2f58e4","Type":"ContainerStarted","Data":"718d2f030af23bf27e06a146162b928fb232462c21fb74fa481368feeec63641"} Jul 12 18:56:02.342254 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:02.342182 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" podStartSLOduration=1.342156423 podStartE2EDuration="1.342156423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:56:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:56:02.339322347 +0000 UTC m=+1853.538789388" watchObservedRunningTime="2026-07-12 18:56:02.342156423 +0000 UTC m=+1853.541623456" Jul 12 18:56:02.580368 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:02.580275 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:02.585649 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:02.585616 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:03.325437 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:03.325397 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:03.327058 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:03.327032 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:07.354639 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:07.354604 2585 generic.go:356] "Generic (PLEG): container finished" podID="574dea44-9ecf-4c1b-abba-526e442b081d" containerID="aba2ffd3c3142754db8861eb57176046008cc17131a4a4c5716ca70fd3721e98" exitCode=0 Jul 12 18:56:07.354639 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:07.354645 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" event={"ID":"574dea44-9ecf-4c1b-abba-526e442b081d","Type":"ContainerDied","Data":"aba2ffd3c3142754db8861eb57176046008cc17131a4a4c5716ca70fd3721e98"} Jul 12 18:56:07.355157 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:07.355041 2585 scope.go:117] "RemoveContainer" containerID="aba2ffd3c3142754db8861eb57176046008cc17131a4a4c5716ca70fd3721e98" Jul 12 18:56:07.356107 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:07.356073 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:56:07.356228 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:07.356169 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:56:07.356294 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:07.356227 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:56:08.362730 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:08.362695 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" event={"ID":"574dea44-9ecf-4c1b-abba-526e442b081d","Type":"ContainerStarted","Data":"11ecebd46f728b8572f840ff84eafebbf9f985a7ced69d28145a6ae86867017c"} Jul 12 18:56:09.295402 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:09.295364 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:56:09.296636 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:09.296609 2585 patch_prober.go:28] interesting pod/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 12 18:56:09.296744 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:09.296669 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" podUID="574dea44-9ecf-4c1b-abba-526e442b081d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 12 18:56:10.277903 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:10.277847 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:56:10.294879 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:10.294846 2585 patch_prober.go:28] interesting pod/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 12 18:56:10.295078 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:10.294898 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" podUID="574dea44-9ecf-4c1b-abba-526e442b081d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 12 18:56:11.211391 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:11.211340 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:56:11.295034 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:11.294996 2585 patch_prober.go:28] interesting pod/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 12 18:56:11.295435 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:11.295055 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" podUID="574dea44-9ecf-4c1b-abba-526e442b081d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 12 18:56:12.294944 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:12.294901 2585 patch_prober.go:28] interesting pod/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 12 18:56:12.295148 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:12.295017 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" podUID="574dea44-9ecf-4c1b-abba-526e442b081d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 12 18:56:13.304455 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:13.304426 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:56:13.305549 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:13.305529 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:56:18.902183 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:18.902150 2585 patch_prober.go:28] interesting pod/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": read tcp 10.132.0.2:35094->10.132.0.53:15021: read: connection reset by peer" start-of-body= Jul 12 18:56:18.902733 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:18.902695 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" podUID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": read tcp 10.132.0.2:35094->10.132.0.53:15021: read: connection reset by peer" Jul 12 18:56:19.428168 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:19.428136 2585 generic.go:356] "Generic (PLEG): container finished" podID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" containerID="1fa58b71d4298ff625ad90e114bdf7492d481d354cb7427bfbb20297c1df6dd9" exitCode=0 Jul 12 18:56:19.428355 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:19.428222 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" event={"ID":"31d9f1be-4d1e-4b02-926b-07829e2f58e4","Type":"ContainerDied","Data":"1fa58b71d4298ff625ad90e114bdf7492d481d354cb7427bfbb20297c1df6dd9"} Jul 12 18:56:19.428623 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:19.428608 2585 scope.go:117] "RemoveContainer" containerID="1fa58b71d4298ff625ad90e114bdf7492d481d354cb7427bfbb20297c1df6dd9" Jul 12 18:56:19.429684 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:19.429645 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:56:19.429786 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:19.429739 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:56:19.429849 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:19.429791 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:56:20.440665 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:20.440622 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" event={"ID":"31d9f1be-4d1e-4b02-926b-07829e2f58e4","Type":"ContainerStarted","Data":"20147e442e8ff916c33f2286f90a72f6361dc2812c2d3ecbb5d814baaf190390"} Jul 12 18:56:20.580165 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:20.580124 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:20.580461 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:20.580438 2585 patch_prober.go:28] interesting pod/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 12 18:56:20.580507 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:20.580487 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" podUID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 12 18:56:21.580951 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:21.580891 2585 patch_prober.go:28] interesting pod/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 12 18:56:21.581368 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:21.580974 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" podUID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 12 18:56:22.580659 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:22.580625 2585 patch_prober.go:28] interesting pod/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 12 18:56:22.580848 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:22.580681 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" podUID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 12 18:56:22.875971 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:22.875852 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:56:23.580730 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:23.580693 2585 patch_prober.go:28] interesting pod/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 12 18:56:23.580969 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:23.580746 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" podUID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 12 18:56:24.584402 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:24.584360 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:24.584791 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:24.584628 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:24.585336 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:24.585318 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:56:50.217583 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.217476 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz"] Jul 12 18:56:50.221230 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.221209 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.224985 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.224953 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-47ad52\"/\"default-dockercfg-vbxnw\"" Jul 12 18:56:50.225375 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.225075 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-47ad52\"/\"openshift-service-ca.crt\"" Jul 12 18:56:50.225631 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.225103 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-47ad52\"/\"test-model-47ad52-kserve-self-signed-certs\"" Jul 12 18:56:50.225774 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.225128 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-47ad52\"/\"kube-root-ca.crt\"" Jul 12 18:56:50.234735 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.234707 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz"] Jul 12 18:56:50.321827 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.321768 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5n2b\" (UniqueName: \"kubernetes.io/projected/2594b72d-af16-4044-a3c1-435b25117152-kube-api-access-l5n2b\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.321827 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.321827 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-home\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.322100 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.321976 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-model-cache\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.322100 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.322011 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2594b72d-af16-4044-a3c1-435b25117152-tls-certs\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.322100 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.322037 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-tmp-dir\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.322100 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.322060 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-dshm\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.423180 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.423141 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-dshm\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.423372 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.423220 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5n2b\" (UniqueName: \"kubernetes.io/projected/2594b72d-af16-4044-a3c1-435b25117152-kube-api-access-l5n2b\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.423372 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.423240 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-home\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.423372 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.423292 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-model-cache\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.423372 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.423312 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2594b72d-af16-4044-a3c1-435b25117152-tls-certs\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.423558 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.423439 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-tmp-dir\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.423718 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.423692 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-home\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.423806 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.423727 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-model-cache\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.423806 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.423786 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-tmp-dir\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.425508 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.425489 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-dshm\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.425803 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.425786 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2594b72d-af16-4044-a3c1-435b25117152-tls-certs\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.432127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.432097 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5n2b\" (UniqueName: \"kubernetes.io/projected/2594b72d-af16-4044-a3c1-435b25117152-kube-api-access-l5n2b\") pod \"test-model-47ad52-kserve-894fdd49f-2rlkz\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.533107 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.533055 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:56:50.678356 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:50.678326 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz"] Jul 12 18:56:50.680319 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:56:50.680281 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2594b72d_af16_4044_a3c1_435b25117152.slice/crio-2a25fc3f93c5bdc9c0b2382970cc5f2b28ea17ca385623869f6762645f5e0934 WatchSource:0}: Error finding container 2a25fc3f93c5bdc9c0b2382970cc5f2b28ea17ca385623869f6762645f5e0934: Status 404 returned error can't find the container with id 2a25fc3f93c5bdc9c0b2382970cc5f2b28ea17ca385623869f6762645f5e0934 Jul 12 18:56:51.626079 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:51.626040 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" event={"ID":"2594b72d-af16-4044-a3c1-435b25117152","Type":"ContainerStarted","Data":"8eeb384c14f33bcad541bedea4d1d697a7396def97f19acdd6df9fd4acf4b9f6"} Jul 12 18:56:51.626079 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:51.626083 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" event={"ID":"2594b72d-af16-4044-a3c1-435b25117152","Type":"ContainerStarted","Data":"2a25fc3f93c5bdc9c0b2382970cc5f2b28ea17ca385623869f6762645f5e0934"} Jul 12 18:56:51.643769 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:51.643713 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" podStartSLOduration=1.643695058 podStartE2EDuration="1.643695058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:56:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:56:51.642217489 +0000 UTC m=+1902.841684519" watchObservedRunningTime="2026-07-12 18:56:51.643695058 +0000 UTC m=+1902.843162119" Jul 12 18:56:54.013253 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:56:54.013208 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:57:00.533548 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:00.533506 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:57:00.534198 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:00.533601 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:57:00.546584 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:00.546553 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:57:00.687668 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:00.687632 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:57:08.103884 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:08.103848 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:57:13.425251 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:13.425210 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz"] Jul 12 18:57:23.012320 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:23.012278 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:57:28.447303 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.447235 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" podUID="2594b72d-af16-4044-a3c1-435b25117152" containerName="main" containerID="cri-o://8eeb384c14f33bcad541bedea4d1d697a7396def97f19acdd6df9fd4acf4b9f6" gracePeriod=45 Jul 12 18:57:28.710153 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.710080 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:57:28.781081 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.781036 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-home\") pod \"2594b72d-af16-4044-a3c1-435b25117152\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " Jul 12 18:57:28.781262 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.781101 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-dshm\") pod \"2594b72d-af16-4044-a3c1-435b25117152\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " Jul 12 18:57:28.781262 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.781146 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-model-cache\") pod \"2594b72d-af16-4044-a3c1-435b25117152\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " Jul 12 18:57:28.781262 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.781200 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2594b72d-af16-4044-a3c1-435b25117152-tls-certs\") pod \"2594b72d-af16-4044-a3c1-435b25117152\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " Jul 12 18:57:28.781262 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.781224 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-tmp-dir\") pod \"2594b72d-af16-4044-a3c1-435b25117152\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " Jul 12 18:57:28.781469 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.781319 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5n2b\" (UniqueName: \"kubernetes.io/projected/2594b72d-af16-4044-a3c1-435b25117152-kube-api-access-l5n2b\") pod \"2594b72d-af16-4044-a3c1-435b25117152\" (UID: \"2594b72d-af16-4044-a3c1-435b25117152\") " Jul 12 18:57:28.781469 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.781420 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-home" (OuterVolumeSpecName: "home") pod "2594b72d-af16-4044-a3c1-435b25117152" (UID: "2594b72d-af16-4044-a3c1-435b25117152"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:57:28.781580 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.781445 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-model-cache" (OuterVolumeSpecName: "model-cache") pod "2594b72d-af16-4044-a3c1-435b25117152" (UID: "2594b72d-af16-4044-a3c1-435b25117152"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:57:28.781637 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.781578 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2594b72d-af16-4044-a3c1-435b25117152" (UID: "2594b72d-af16-4044-a3c1-435b25117152"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:57:28.781711 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.781693 2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-home\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:57:28.781766 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.781716 2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-model-cache\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:57:28.781766 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.781732 2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-tmp-dir\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:57:28.783539 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.783496 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2594b72d-af16-4044-a3c1-435b25117152-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2594b72d-af16-4044-a3c1-435b25117152" (UID: "2594b72d-af16-4044-a3c1-435b25117152"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:57:28.783539 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.783508 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-dshm" (OuterVolumeSpecName: "dshm") pod "2594b72d-af16-4044-a3c1-435b25117152" (UID: "2594b72d-af16-4044-a3c1-435b25117152"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:57:28.783701 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.783572 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2594b72d-af16-4044-a3c1-435b25117152-kube-api-access-l5n2b" (OuterVolumeSpecName: "kube-api-access-l5n2b") pod "2594b72d-af16-4044-a3c1-435b25117152" (UID: "2594b72d-af16-4044-a3c1-435b25117152"). InnerVolumeSpecName "kube-api-access-l5n2b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:57:28.832097 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.832066 2585 generic.go:356] "Generic (PLEG): container finished" podID="2594b72d-af16-4044-a3c1-435b25117152" containerID="8eeb384c14f33bcad541bedea4d1d697a7396def97f19acdd6df9fd4acf4b9f6" exitCode=0 Jul 12 18:57:28.832272 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.832107 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" Jul 12 18:57:28.832272 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.832152 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" event={"ID":"2594b72d-af16-4044-a3c1-435b25117152","Type":"ContainerDied","Data":"8eeb384c14f33bcad541bedea4d1d697a7396def97f19acdd6df9fd4acf4b9f6"} Jul 12 18:57:28.832272 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.832203 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz" event={"ID":"2594b72d-af16-4044-a3c1-435b25117152","Type":"ContainerDied","Data":"2a25fc3f93c5bdc9c0b2382970cc5f2b28ea17ca385623869f6762645f5e0934"} Jul 12 18:57:28.832272 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.832227 2585 scope.go:117] "RemoveContainer" containerID="8eeb384c14f33bcad541bedea4d1d697a7396def97f19acdd6df9fd4acf4b9f6" Jul 12 18:57:28.843180 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.843162 2585 scope.go:117] "RemoveContainer" containerID="8eeb384c14f33bcad541bedea4d1d697a7396def97f19acdd6df9fd4acf4b9f6" Jul 12 18:57:28.843542 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:57:28.843516 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eeb384c14f33bcad541bedea4d1d697a7396def97f19acdd6df9fd4acf4b9f6\": container with ID starting with 8eeb384c14f33bcad541bedea4d1d697a7396def97f19acdd6df9fd4acf4b9f6 not found: ID does not exist" containerID="8eeb384c14f33bcad541bedea4d1d697a7396def97f19acdd6df9fd4acf4b9f6" Jul 12 18:57:28.843629 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.843554 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eeb384c14f33bcad541bedea4d1d697a7396def97f19acdd6df9fd4acf4b9f6"} err="failed to get container status \"8eeb384c14f33bcad541bedea4d1d697a7396def97f19acdd6df9fd4acf4b9f6\": rpc error: code = NotFound desc = could not find container \"8eeb384c14f33bcad541bedea4d1d697a7396def97f19acdd6df9fd4acf4b9f6\": container with ID starting with 8eeb384c14f33bcad541bedea4d1d697a7396def97f19acdd6df9fd4acf4b9f6 not found: ID does not exist" Jul 12 18:57:28.854101 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.854073 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz"] Jul 12 18:57:28.858014 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.857987 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-47ad52/test-model-47ad52-kserve-894fdd49f-2rlkz"] Jul 12 18:57:28.883168 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.883136 2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2594b72d-af16-4044-a3c1-435b25117152-dshm\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:57:28.883168 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.883172 2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2594b72d-af16-4044-a3c1-435b25117152-tls-certs\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:57:28.883371 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:28.883184 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5n2b\" (UniqueName: \"kubernetes.io/projected/2594b72d-af16-4044-a3c1-435b25117152-kube-api-access-l5n2b\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:57:29.305093 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:29.305056 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2594b72d-af16-4044-a3c1-435b25117152" path="/var/lib/kubelet/pods/2594b72d-af16-4044-a3c1-435b25117152/volumes" Jul 12 18:57:55.961700 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:55.961655 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg"] Jul 12 18:57:55.962217 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:55.961955 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" podUID="574dea44-9ecf-4c1b-abba-526e442b081d" containerName="istio-proxy" containerID="cri-o://11ecebd46f728b8572f840ff84eafebbf9f985a7ced69d28145a6ae86867017c" gracePeriod=30 Jul 12 18:57:58.307630 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:58.307584 2585 patch_prober.go:28] interesting pod/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 12 18:57:58.308040 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:57:58.307655 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" podUID="574dea44-9ecf-4c1b-abba-526e442b081d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 18:58:01.223389 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.223322 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:58:01.280166 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280128 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-workload-certs\") pod \"574dea44-9ecf-4c1b-abba-526e442b081d\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " Jul 12 18:58:01.280166 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280174 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/574dea44-9ecf-4c1b-abba-526e442b081d-istiod-ca-cert\") pod \"574dea44-9ecf-4c1b-abba-526e442b081d\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " Jul 12 18:58:01.280380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280202 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hb4q8\" (UniqueName: \"kubernetes.io/projected/574dea44-9ecf-4c1b-abba-526e442b081d-kube-api-access-hb4q8\") pod \"574dea44-9ecf-4c1b-abba-526e442b081d\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " Jul 12 18:58:01.280380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280276 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-istio-data\") pod \"574dea44-9ecf-4c1b-abba-526e442b081d\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " Jul 12 18:58:01.280380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280326 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-credential-socket\") pod \"574dea44-9ecf-4c1b-abba-526e442b081d\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " Jul 12 18:58:01.280380 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280368 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/574dea44-9ecf-4c1b-abba-526e442b081d-istio-token\") pod \"574dea44-9ecf-4c1b-abba-526e442b081d\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " Jul 12 18:58:01.280578 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280391 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-workload-socket\") pod \"574dea44-9ecf-4c1b-abba-526e442b081d\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " Jul 12 18:58:01.280578 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280467 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/574dea44-9ecf-4c1b-abba-526e442b081d-istio-podinfo\") pod \"574dea44-9ecf-4c1b-abba-526e442b081d\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " Jul 12 18:58:01.280578 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280497 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-istio-envoy\") pod \"574dea44-9ecf-4c1b-abba-526e442b081d\" (UID: \"574dea44-9ecf-4c1b-abba-526e442b081d\") " Jul 12 18:58:01.280578 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280473 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "574dea44-9ecf-4c1b-abba-526e442b081d" (UID: "574dea44-9ecf-4c1b-abba-526e442b081d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:01.280782 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280620 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/574dea44-9ecf-4c1b-abba-526e442b081d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "574dea44-9ecf-4c1b-abba-526e442b081d" (UID: "574dea44-9ecf-4c1b-abba-526e442b081d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:58:01.280846 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280801 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-workload-certs\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:01.280846 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280825 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/574dea44-9ecf-4c1b-abba-526e442b081d-istiod-ca-cert\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:01.280846 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280831 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "574dea44-9ecf-4c1b-abba-526e442b081d" (UID: "574dea44-9ecf-4c1b-abba-526e442b081d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:01.281051 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.280897 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "574dea44-9ecf-4c1b-abba-526e442b081d" (UID: "574dea44-9ecf-4c1b-abba-526e442b081d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:01.281251 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.281224 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-istio-data" (OuterVolumeSpecName: "istio-data") pod "574dea44-9ecf-4c1b-abba-526e442b081d" (UID: "574dea44-9ecf-4c1b-abba-526e442b081d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:01.282614 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.282583 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/574dea44-9ecf-4c1b-abba-526e442b081d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "574dea44-9ecf-4c1b-abba-526e442b081d" (UID: "574dea44-9ecf-4c1b-abba-526e442b081d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:58:01.282720 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.282608 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/574dea44-9ecf-4c1b-abba-526e442b081d-istio-token" (OuterVolumeSpecName: "istio-token") pod "574dea44-9ecf-4c1b-abba-526e442b081d" (UID: "574dea44-9ecf-4c1b-abba-526e442b081d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:58:01.282720 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.282640 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "574dea44-9ecf-4c1b-abba-526e442b081d" (UID: "574dea44-9ecf-4c1b-abba-526e442b081d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:01.282720 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.282696 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/574dea44-9ecf-4c1b-abba-526e442b081d-kube-api-access-hb4q8" (OuterVolumeSpecName: "kube-api-access-hb4q8") pod "574dea44-9ecf-4c1b-abba-526e442b081d" (UID: "574dea44-9ecf-4c1b-abba-526e442b081d"). InnerVolumeSpecName "kube-api-access-hb4q8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:58:01.381800 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.381717 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb4q8\" (UniqueName: \"kubernetes.io/projected/574dea44-9ecf-4c1b-abba-526e442b081d-kube-api-access-hb4q8\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:01.381800 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.381750 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-istio-data\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:01.381800 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.381759 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-credential-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:01.381800 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.381770 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/574dea44-9ecf-4c1b-abba-526e442b081d-istio-token\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:01.381800 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.381779 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-workload-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:01.381800 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.381787 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/574dea44-9ecf-4c1b-abba-526e442b081d-istio-podinfo\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:01.381800 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:01.381796 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/574dea44-9ecf-4c1b-abba-526e442b081d-istio-envoy\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:02.022495 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:02.022458 2585 generic.go:356] "Generic (PLEG): container finished" podID="574dea44-9ecf-4c1b-abba-526e442b081d" containerID="11ecebd46f728b8572f840ff84eafebbf9f985a7ced69d28145a6ae86867017c" exitCode=0 Jul 12 18:58:02.022715 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:02.022511 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" Jul 12 18:58:02.022715 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:02.022542 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" event={"ID":"574dea44-9ecf-4c1b-abba-526e442b081d","Type":"ContainerDied","Data":"11ecebd46f728b8572f840ff84eafebbf9f985a7ced69d28145a6ae86867017c"} Jul 12 18:58:02.022715 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:02.022611 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg" event={"ID":"574dea44-9ecf-4c1b-abba-526e442b081d","Type":"ContainerDied","Data":"b1857841b63428d577b190b139edf7e5694cf51285a9a3229417ceeab6265d0c"} Jul 12 18:58:02.022715 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:02.022653 2585 scope.go:117] "RemoveContainer" containerID="11ecebd46f728b8572f840ff84eafebbf9f985a7ced69d28145a6ae86867017c" Jul 12 18:58:02.038134 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:02.038111 2585 scope.go:117] "RemoveContainer" containerID="aba2ffd3c3142754db8861eb57176046008cc17131a4a4c5716ca70fd3721e98" Jul 12 18:58:02.045276 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:02.045245 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg"] Jul 12 18:58:02.049862 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:02.049839 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-58790d-openshift-default-6bc9f6c867-vkxjg"] Jul 12 18:58:02.049938 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:02.049881 2585 scope.go:117] "RemoveContainer" containerID="11ecebd46f728b8572f840ff84eafebbf9f985a7ced69d28145a6ae86867017c" Jul 12 18:58:02.050231 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:58:02.050207 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11ecebd46f728b8572f840ff84eafebbf9f985a7ced69d28145a6ae86867017c\": container with ID starting with 11ecebd46f728b8572f840ff84eafebbf9f985a7ced69d28145a6ae86867017c not found: ID does not exist" containerID="11ecebd46f728b8572f840ff84eafebbf9f985a7ced69d28145a6ae86867017c" Jul 12 18:58:02.050275 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:02.050241 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11ecebd46f728b8572f840ff84eafebbf9f985a7ced69d28145a6ae86867017c"} err="failed to get container status \"11ecebd46f728b8572f840ff84eafebbf9f985a7ced69d28145a6ae86867017c\": rpc error: code = NotFound desc = could not find container \"11ecebd46f728b8572f840ff84eafebbf9f985a7ced69d28145a6ae86867017c\": container with ID starting with 11ecebd46f728b8572f840ff84eafebbf9f985a7ced69d28145a6ae86867017c not found: ID does not exist" Jul 12 18:58:02.050275 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:02.050259 2585 scope.go:117] "RemoveContainer" containerID="aba2ffd3c3142754db8861eb57176046008cc17131a4a4c5716ca70fd3721e98" Jul 12 18:58:02.050544 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:58:02.050525 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aba2ffd3c3142754db8861eb57176046008cc17131a4a4c5716ca70fd3721e98\": container with ID starting with aba2ffd3c3142754db8861eb57176046008cc17131a4a4c5716ca70fd3721e98 not found: ID does not exist" containerID="aba2ffd3c3142754db8861eb57176046008cc17131a4a4c5716ca70fd3721e98" Jul 12 18:58:02.050611 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:02.050563 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aba2ffd3c3142754db8861eb57176046008cc17131a4a4c5716ca70fd3721e98"} err="failed to get container status \"aba2ffd3c3142754db8861eb57176046008cc17131a4a4c5716ca70fd3721e98\": rpc error: code = NotFound desc = could not find container \"aba2ffd3c3142754db8861eb57176046008cc17131a4a4c5716ca70fd3721e98\": container with ID starting with aba2ffd3c3142754db8861eb57176046008cc17131a4a4c5716ca70fd3721e98 not found: ID does not exist" Jul 12 18:58:03.115361 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.115319 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6"] Jul 12 18:58:03.115787 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.115570 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" podUID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" containerName="istio-proxy" containerID="cri-o://20147e442e8ff916c33f2286f90a72f6361dc2812c2d3ecbb5d814baaf190390" gracePeriod=30 Jul 12 18:58:03.304832 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.304795 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="574dea44-9ecf-4c1b-abba-526e442b081d" path="/var/lib/kubelet/pods/574dea44-9ecf-4c1b-abba-526e442b081d/volumes" Jul 12 18:58:03.323608 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.323574 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:58:03.990447 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.990408 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq"] Jul 12 18:58:03.990871 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.990852 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="574dea44-9ecf-4c1b-abba-526e442b081d" containerName="istio-proxy" Jul 12 18:58:03.990977 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.990873 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="574dea44-9ecf-4c1b-abba-526e442b081d" containerName="istio-proxy" Jul 12 18:58:03.990977 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.990893 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="574dea44-9ecf-4c1b-abba-526e442b081d" containerName="istio-proxy" Jul 12 18:58:03.990977 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.990902 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="574dea44-9ecf-4c1b-abba-526e442b081d" containerName="istio-proxy" Jul 12 18:58:03.990977 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.990949 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2594b72d-af16-4044-a3c1-435b25117152" containerName="main" Jul 12 18:58:03.990977 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.990960 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2594b72d-af16-4044-a3c1-435b25117152" containerName="main" Jul 12 18:58:03.991234 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.991057 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="574dea44-9ecf-4c1b-abba-526e442b081d" containerName="istio-proxy" Jul 12 18:58:03.991234 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.991071 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2594b72d-af16-4044-a3c1-435b25117152" containerName="main" Jul 12 18:58:03.991234 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.991083 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="574dea44-9ecf-4c1b-abba-526e442b081d" containerName="istio-proxy" Jul 12 18:58:03.995790 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.995761 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:03.999765 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:03.999733 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-6ee31b-openshift-default-dockercfg-gqhf8\"" Jul 12 18:58:04.013343 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.013307 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq"] Jul 12 18:58:04.110320 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.110282 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-envoy\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.110320 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.110320 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff04a479-3205-4407-ae9e-c9a00a8d8966-istiod-ca-cert\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.110552 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.110350 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-podinfo\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.110552 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.110383 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-workload-certs\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.110552 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.110418 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq4gm\" (UniqueName: \"kubernetes.io/projected/ff04a479-3205-4407-ae9e-c9a00a8d8966-kube-api-access-gq4gm\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.110552 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.110472 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-data\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.110552 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.110511 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-workload-socket\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.110552 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.110530 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-credential-socket\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.110776 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.110573 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-token\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.211785 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.211746 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-envoy\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.211785 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.211795 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff04a479-3205-4407-ae9e-c9a00a8d8966-istiod-ca-cert\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.212355 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.211828 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-podinfo\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.212355 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.211868 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-workload-certs\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.212355 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.212073 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq4gm\" (UniqueName: \"kubernetes.io/projected/ff04a479-3205-4407-ae9e-c9a00a8d8966-kube-api-access-gq4gm\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.212355 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.212147 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-data\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.212355 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.212202 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-workload-socket\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.212355 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.212228 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-credential-socket\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.212355 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.212290 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-workload-certs\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.212355 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.212302 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-token\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.212775 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.212658 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-data\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.212775 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.212699 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-workload-socket\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.212913 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.212890 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-credential-socket\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.213303 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.213277 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff04a479-3205-4407-ae9e-c9a00a8d8966-istiod-ca-cert\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.214550 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.214520 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-envoy\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.214644 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.214608 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-podinfo\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.220809 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.220782 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-token\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.220923 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.220906 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq4gm\" (UniqueName: \"kubernetes.io/projected/ff04a479-3205-4407-ae9e-c9a00a8d8966-kube-api-access-gq4gm\") pod \"e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.306391 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.306358 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:04.458607 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.458575 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq"] Jul 12 18:58:04.460459 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:58:04.460428 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff04a479_3205_4407_ae9e_c9a00a8d8966.slice/crio-2a70c8c19849cda819d6c301244931cde5c32490de43a861dd8ad95ea02e2ab5 WatchSource:0}: Error finding container 2a70c8c19849cda819d6c301244931cde5c32490de43a861dd8ad95ea02e2ab5: Status 404 returned error can't find the container with id 2a70c8c19849cda819d6c301244931cde5c32490de43a861dd8ad95ea02e2ab5 Jul 12 18:58:04.462605 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.462571 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:04.462692 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.462642 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:04.462692 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:04.462674 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:05.043557 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:05.043520 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" event={"ID":"ff04a479-3205-4407-ae9e-c9a00a8d8966","Type":"ContainerStarted","Data":"925f352dbf56a07db83513fdf874ec4816de1bef098d9ab2c7ec33adc9eef1db"} Jul 12 18:58:05.043737 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:05.043579 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" event={"ID":"ff04a479-3205-4407-ae9e-c9a00a8d8966","Type":"ContainerStarted","Data":"2a70c8c19849cda819d6c301244931cde5c32490de43a861dd8ad95ea02e2ab5"} Jul 12 18:58:05.068886 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:05.068833 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" podStartSLOduration=2.068815013 podStartE2EDuration="2.068815013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:58:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:58:05.065893646 +0000 UTC m=+1976.265360679" watchObservedRunningTime="2026-07-12 18:58:05.068815013 +0000 UTC m=+1976.268282044" Jul 12 18:58:05.307494 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:05.307414 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:05.312548 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:05.312514 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:06.049759 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:06.049725 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:06.051022 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:06.050995 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:08.380823 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.380744 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:58:08.456044 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456007 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istiod-ca-cert\") pod \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " Jul 12 18:58:08.456258 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456058 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9cmm\" (UniqueName: \"kubernetes.io/projected/31d9f1be-4d1e-4b02-926b-07829e2f58e4-kube-api-access-h9cmm\") pod \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " Jul 12 18:58:08.456258 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456078 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-envoy\") pod \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " Jul 12 18:58:08.456258 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456099 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-token\") pod \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " Jul 12 18:58:08.456258 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456142 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-workload-certs\") pod \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " Jul 12 18:58:08.456258 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456178 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-podinfo\") pod \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " Jul 12 18:58:08.456258 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456222 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-workload-socket\") pod \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " Jul 12 18:58:08.456581 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456282 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-data\") pod \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " Jul 12 18:58:08.456581 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456324 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-credential-socket\") pod \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\" (UID: \"31d9f1be-4d1e-4b02-926b-07829e2f58e4\") " Jul 12 18:58:08.456581 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456534 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "31d9f1be-4d1e-4b02-926b-07829e2f58e4" (UID: "31d9f1be-4d1e-4b02-926b-07829e2f58e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:08.456727 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456586 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "31d9f1be-4d1e-4b02-926b-07829e2f58e4" (UID: "31d9f1be-4d1e-4b02-926b-07829e2f58e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:08.456727 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456598 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "31d9f1be-4d1e-4b02-926b-07829e2f58e4" (UID: "31d9f1be-4d1e-4b02-926b-07829e2f58e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:58:08.456836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456726 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-workload-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:08.456836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456750 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istiod-ca-cert\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:08.456836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456767 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-workload-certs\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:08.456836 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.456771 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "31d9f1be-4d1e-4b02-926b-07829e2f58e4" (UID: "31d9f1be-4d1e-4b02-926b-07829e2f58e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:08.457486 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.457453 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-data" (OuterVolumeSpecName: "istio-data") pod "31d9f1be-4d1e-4b02-926b-07829e2f58e4" (UID: "31d9f1be-4d1e-4b02-926b-07829e2f58e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:08.458468 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.458438 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "31d9f1be-4d1e-4b02-926b-07829e2f58e4" (UID: "31d9f1be-4d1e-4b02-926b-07829e2f58e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:08.458581 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.458494 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "31d9f1be-4d1e-4b02-926b-07829e2f58e4" (UID: "31d9f1be-4d1e-4b02-926b-07829e2f58e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:58:08.458581 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.458534 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "31d9f1be-4d1e-4b02-926b-07829e2f58e4" (UID: "31d9f1be-4d1e-4b02-926b-07829e2f58e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:58:08.458814 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.458785 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31d9f1be-4d1e-4b02-926b-07829e2f58e4-kube-api-access-h9cmm" (OuterVolumeSpecName: "kube-api-access-h9cmm") pod "31d9f1be-4d1e-4b02-926b-07829e2f58e4" (UID: "31d9f1be-4d1e-4b02-926b-07829e2f58e4"). InnerVolumeSpecName "kube-api-access-h9cmm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:58:08.557990 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.557949 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-token\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:08.557990 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.557983 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-podinfo\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:08.557990 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.557994 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-data\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:08.558239 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.558006 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-credential-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:08.558239 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.558019 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9cmm\" (UniqueName: \"kubernetes.io/projected/31d9f1be-4d1e-4b02-926b-07829e2f58e4-kube-api-access-h9cmm\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:08.558239 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:08.558030 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31d9f1be-4d1e-4b02-926b-07829e2f58e4-istio-envoy\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:09.067743 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:09.067710 2585 generic.go:356] "Generic (PLEG): container finished" podID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" containerID="20147e442e8ff916c33f2286f90a72f6361dc2812c2d3ecbb5d814baaf190390" exitCode=0 Jul 12 18:58:09.067923 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:09.067762 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" Jul 12 18:58:09.067923 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:09.067780 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" event={"ID":"31d9f1be-4d1e-4b02-926b-07829e2f58e4","Type":"ContainerDied","Data":"20147e442e8ff916c33f2286f90a72f6361dc2812c2d3ecbb5d814baaf190390"} Jul 12 18:58:09.067923 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:09.067824 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6" event={"ID":"31d9f1be-4d1e-4b02-926b-07829e2f58e4","Type":"ContainerDied","Data":"718d2f030af23bf27e06a146162b928fb232462c21fb74fa481368feeec63641"} Jul 12 18:58:09.067923 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:09.067847 2585 scope.go:117] "RemoveContainer" containerID="20147e442e8ff916c33f2286f90a72f6361dc2812c2d3ecbb5d814baaf190390" Jul 12 18:58:09.078296 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:09.078276 2585 scope.go:117] "RemoveContainer" containerID="1fa58b71d4298ff625ad90e114bdf7492d481d354cb7427bfbb20297c1df6dd9" Jul 12 18:58:09.087448 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:09.087428 2585 scope.go:117] "RemoveContainer" containerID="20147e442e8ff916c33f2286f90a72f6361dc2812c2d3ecbb5d814baaf190390" Jul 12 18:58:09.087772 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:58:09.087747 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20147e442e8ff916c33f2286f90a72f6361dc2812c2d3ecbb5d814baaf190390\": container with ID starting with 20147e442e8ff916c33f2286f90a72f6361dc2812c2d3ecbb5d814baaf190390 not found: ID does not exist" containerID="20147e442e8ff916c33f2286f90a72f6361dc2812c2d3ecbb5d814baaf190390" Jul 12 18:58:09.087845 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:09.087784 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20147e442e8ff916c33f2286f90a72f6361dc2812c2d3ecbb5d814baaf190390"} err="failed to get container status \"20147e442e8ff916c33f2286f90a72f6361dc2812c2d3ecbb5d814baaf190390\": rpc error: code = NotFound desc = could not find container \"20147e442e8ff916c33f2286f90a72f6361dc2812c2d3ecbb5d814baaf190390\": container with ID starting with 20147e442e8ff916c33f2286f90a72f6361dc2812c2d3ecbb5d814baaf190390 not found: ID does not exist" Jul 12 18:58:09.087845 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:09.087803 2585 scope.go:117] "RemoveContainer" containerID="1fa58b71d4298ff625ad90e114bdf7492d481d354cb7427bfbb20297c1df6dd9" Jul 12 18:58:09.088146 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:58:09.088121 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fa58b71d4298ff625ad90e114bdf7492d481d354cb7427bfbb20297c1df6dd9\": container with ID starting with 1fa58b71d4298ff625ad90e114bdf7492d481d354cb7427bfbb20297c1df6dd9 not found: ID does not exist" containerID="1fa58b71d4298ff625ad90e114bdf7492d481d354cb7427bfbb20297c1df6dd9" Jul 12 18:58:09.088227 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:09.088149 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fa58b71d4298ff625ad90e114bdf7492d481d354cb7427bfbb20297c1df6dd9"} err="failed to get container status \"1fa58b71d4298ff625ad90e114bdf7492d481d354cb7427bfbb20297c1df6dd9\": rpc error: code = NotFound desc = could not find container \"1fa58b71d4298ff625ad90e114bdf7492d481d354cb7427bfbb20297c1df6dd9\": container with ID starting with 1fa58b71d4298ff625ad90e114bdf7492d481d354cb7427bfbb20297c1df6dd9 not found: ID does not exist" Jul 12 18:58:09.093747 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:09.093712 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6"] Jul 12 18:58:09.098473 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:09.098438 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-47ad52-openshift-default-dfd86656f-lcjg6"] Jul 12 18:58:09.305323 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:09.305281 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" path="/var/lib/kubelet/pods/31d9f1be-4d1e-4b02-926b-07829e2f58e4/volumes" Jul 12 18:58:12.114712 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.114670 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:58:12.186656 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.186624 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j"] Jul 12 18:58:12.187030 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.187016 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" containerName="istio-proxy" Jul 12 18:58:12.187080 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.187031 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" containerName="istio-proxy" Jul 12 18:58:12.187080 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.187042 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" containerName="istio-proxy" Jul 12 18:58:12.187080 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.187047 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" containerName="istio-proxy" Jul 12 18:58:12.187172 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.187112 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" containerName="istio-proxy" Jul 12 18:58:12.187172 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.187119 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="31d9f1be-4d1e-4b02-926b-07829e2f58e4" containerName="istio-proxy" Jul 12 18:58:12.191918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.191893 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.194287 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.194264 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-498f3f-openshift-default-dockercfg-7dtf8\"" Jul 12 18:58:12.202462 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.202433 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j"] Jul 12 18:58:12.293709 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.293658 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780038ad-a845-4003-95f0-e5e36fcec5d8-istiod-ca-cert\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.293902 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.293719 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-envoy\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.293902 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.293797 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-workload-certs\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.293902 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.293827 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-data\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.294132 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.293968 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-credential-socket\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.294132 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.294002 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z54h5\" (UniqueName: \"kubernetes.io/projected/780038ad-a845-4003-95f0-e5e36fcec5d8-kube-api-access-z54h5\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.294132 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.294040 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-token\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.294132 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.294058 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-podinfo\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.294132 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.294076 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-workload-socket\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.395216 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.395112 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-credential-socket\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.395216 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.395161 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z54h5\" (UniqueName: \"kubernetes.io/projected/780038ad-a845-4003-95f0-e5e36fcec5d8-kube-api-access-z54h5\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.395216 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.395187 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-token\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.395216 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.395203 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-podinfo\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.395216 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.395220 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-workload-socket\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.395610 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.395254 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780038ad-a845-4003-95f0-e5e36fcec5d8-istiod-ca-cert\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.395610 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.395275 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-envoy\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.395610 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.395310 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-workload-certs\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.395610 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.395342 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-data\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.395803 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.395613 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-credential-socket\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.395803 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.395670 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-workload-socket\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.395803 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.395755 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-data\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.395903 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.395855 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-workload-certs\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.396066 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.396047 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780038ad-a845-4003-95f0-e5e36fcec5d8-istiod-ca-cert\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.397710 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.397686 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-envoy\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.398010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.397987 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-podinfo\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.402834 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.402804 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-token\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.402982 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.402960 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z54h5\" (UniqueName: \"kubernetes.io/projected/780038ad-a845-4003-95f0-e5e36fcec5d8-kube-api-access-z54h5\") pod \"e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.504238 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.504196 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:12.653103 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.651442 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j"] Jul 12 18:58:12.654886 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:58:12.654855 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod780038ad_a845_4003_95f0_e5e36fcec5d8.slice/crio-e201187385b2a8e5d993e4aad9ef173dd1465b71c14f2770d7ba53f2b70c3699 WatchSource:0}: Error finding container e201187385b2a8e5d993e4aad9ef173dd1465b71c14f2770d7ba53f2b70c3699: Status 404 returned error can't find the container with id e201187385b2a8e5d993e4aad9ef173dd1465b71c14f2770d7ba53f2b70c3699 Jul 12 18:58:12.657634 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.657598 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:12.657733 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.657671 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:12.657733 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:12.657715 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:13.096379 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:13.096328 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" event={"ID":"780038ad-a845-4003-95f0-e5e36fcec5d8","Type":"ContainerStarted","Data":"85bbc825012109b177568c35cb41e705e6a4b265089448d5e521f3089151ee00"} Jul 12 18:58:13.096598 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:13.096409 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" event={"ID":"780038ad-a845-4003-95f0-e5e36fcec5d8","Type":"ContainerStarted","Data":"e201187385b2a8e5d993e4aad9ef173dd1465b71c14f2770d7ba53f2b70c3699"} Jul 12 18:58:13.115786 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:13.115713 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" podStartSLOduration=1.115688305 podStartE2EDuration="1.115688305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:58:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:58:13.113836386 +0000 UTC m=+1984.313303418" watchObservedRunningTime="2026-07-12 18:58:13.115688305 +0000 UTC m=+1984.315155335" Jul 12 18:58:13.505545 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:13.505448 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:13.506059 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:13.506025 2585 patch_prober.go:28] interesting pod/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 18:58:13.506177 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:13.506112 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" podUID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 18:58:14.509308 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:14.509269 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:15.108071 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:15.108040 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:15.109313 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:15.109283 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:21.051496 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:21.051455 2585 patch_prober.go:28] interesting pod/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 12 18:58:21.051918 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:21.051513 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" podUID="ff04a479-3205-4407-ae9e-c9a00a8d8966" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 12 18:58:21.145244 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:21.145210 2585 generic.go:356] "Generic (PLEG): container finished" podID="ff04a479-3205-4407-ae9e-c9a00a8d8966" containerID="925f352dbf56a07db83513fdf874ec4816de1bef098d9ab2c7ec33adc9eef1db" exitCode=0 Jul 12 18:58:21.145424 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:21.145266 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" event={"ID":"ff04a479-3205-4407-ae9e-c9a00a8d8966","Type":"ContainerDied","Data":"925f352dbf56a07db83513fdf874ec4816de1bef098d9ab2c7ec33adc9eef1db"} Jul 12 18:58:21.145652 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:21.145637 2585 scope.go:117] "RemoveContainer" containerID="925f352dbf56a07db83513fdf874ec4816de1bef098d9ab2c7ec33adc9eef1db" Jul 12 18:58:21.146658 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:21.146624 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:21.146716 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:21.146698 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:21.146767 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:21.146733 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:21.703011 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:21.702971 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:58:22.153701 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:22.153656 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" event={"ID":"ff04a479-3205-4407-ae9e-c9a00a8d8966","Type":"ContainerStarted","Data":"79df71b359bc6a7f64995f1f3a14f3ecaae4fd49cb8fb8413d6e369ce2ff1dd4"} Jul 12 18:58:22.306898 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:22.306857 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:22.307167 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:22.307141 2585 patch_prober.go:28] interesting pod/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 12 18:58:22.307252 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:22.307198 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" podUID="ff04a479-3205-4407-ae9e-c9a00a8d8966" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 12 18:58:23.307286 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:23.307249 2585 patch_prober.go:28] interesting pod/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 12 18:58:23.307705 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:23.307316 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" podUID="ff04a479-3205-4407-ae9e-c9a00a8d8966" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 12 18:58:24.307155 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:24.307111 2585 patch_prober.go:28] interesting pod/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 12 18:58:24.307379 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:24.307178 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" podUID="ff04a479-3205-4407-ae9e-c9a00a8d8966" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 12 18:58:25.306825 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:25.306796 2585 patch_prober.go:28] interesting pod/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 12 18:58:25.307107 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:25.307070 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" podUID="ff04a479-3205-4407-ae9e-c9a00a8d8966" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 12 18:58:26.311010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:26.310976 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:26.311430 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:26.311256 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:26.312302 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:26.312279 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:30.110256 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:30.110159 2585 patch_prober.go:28] interesting pod/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 18:58:30.110736 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:30.110241 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" podUID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 18:58:30.213424 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:30.213329 2585 generic.go:356] "Generic (PLEG): container finished" podID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerID="85bbc825012109b177568c35cb41e705e6a4b265089448d5e521f3089151ee00" exitCode=0 Jul 12 18:58:30.213424 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:30.213406 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" event={"ID":"780038ad-a845-4003-95f0-e5e36fcec5d8","Type":"ContainerDied","Data":"85bbc825012109b177568c35cb41e705e6a4b265089448d5e521f3089151ee00"} Jul 12 18:58:30.213866 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:30.213851 2585 scope.go:117] "RemoveContainer" containerID="85bbc825012109b177568c35cb41e705e6a4b265089448d5e521f3089151ee00" Jul 12 18:58:30.214980 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:30.214952 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:30.215055 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:30.215033 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:30.215092 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:30.215066 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:31.220390 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:31.220356 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" event={"ID":"780038ad-a845-4003-95f0-e5e36fcec5d8","Type":"ContainerStarted","Data":"029f1edeb6857d38bd52c0a33edf183c436f0dd0a29da3d7ed99b620ce533fc3"} Jul 12 18:58:31.504754 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:31.504652 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:31.504910 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:31.504884 2585 patch_prober.go:28] interesting pod/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 18:58:31.504980 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:31.504955 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" podUID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 18:58:32.006894 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:32.006855 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:58:32.505210 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:32.505169 2585 patch_prober.go:28] interesting pod/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 18:58:32.505611 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:32.505233 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" podUID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 18:58:33.505299 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:33.505256 2585 patch_prober.go:28] interesting pod/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 18:58:33.505709 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:33.505331 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" podUID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 18:58:34.222291 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:34.222251 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq"] Jul 12 18:58:34.222546 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:34.222518 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" podUID="ff04a479-3205-4407-ae9e-c9a00a8d8966" containerName="istio-proxy" containerID="cri-o://79df71b359bc6a7f64995f1f3a14f3ecaae4fd49cb8fb8413d6e369ce2ff1dd4" gracePeriod=30 Jul 12 18:58:34.505573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:34.505465 2585 patch_prober.go:28] interesting pod/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 12 18:58:34.505573 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:34.505541 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" podUID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 12 18:58:35.508743 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:35.508708 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:35.509156 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:35.509026 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:35.509944 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:35.509909 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:39.291516 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.291482 2585 generic.go:356] "Generic (PLEG): container finished" podID="ff04a479-3205-4407-ae9e-c9a00a8d8966" containerID="79df71b359bc6a7f64995f1f3a14f3ecaae4fd49cb8fb8413d6e369ce2ff1dd4" exitCode=0 Jul 12 18:58:39.292052 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.291557 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" event={"ID":"ff04a479-3205-4407-ae9e-c9a00a8d8966","Type":"ContainerDied","Data":"79df71b359bc6a7f64995f1f3a14f3ecaae4fd49cb8fb8413d6e369ce2ff1dd4"} Jul 12 18:58:39.292052 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.291630 2585 scope.go:117] "RemoveContainer" containerID="925f352dbf56a07db83513fdf874ec4816de1bef098d9ab2c7ec33adc9eef1db" Jul 12 18:58:39.403813 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.403780 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:39.491651 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.491548 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-podinfo\") pod \"ff04a479-3205-4407-ae9e-c9a00a8d8966\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " Jul 12 18:58:39.491651 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.491610 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-data\") pod \"ff04a479-3205-4407-ae9e-c9a00a8d8966\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " Jul 12 18:58:39.491651 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.491634 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-token\") pod \"ff04a479-3205-4407-ae9e-c9a00a8d8966\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " Jul 12 18:58:39.491970 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.491664 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-envoy\") pod \"ff04a479-3205-4407-ae9e-c9a00a8d8966\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " Jul 12 18:58:39.491970 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.491685 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-credential-socket\") pod \"ff04a479-3205-4407-ae9e-c9a00a8d8966\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " Jul 12 18:58:39.491970 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.491724 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-workload-certs\") pod \"ff04a479-3205-4407-ae9e-c9a00a8d8966\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " Jul 12 18:58:39.491970 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.491812 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff04a479-3205-4407-ae9e-c9a00a8d8966-istiod-ca-cert\") pod \"ff04a479-3205-4407-ae9e-c9a00a8d8966\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " Jul 12 18:58:39.491970 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.491838 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-workload-socket\") pod \"ff04a479-3205-4407-ae9e-c9a00a8d8966\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " Jul 12 18:58:39.491970 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.491868 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gq4gm\" (UniqueName: \"kubernetes.io/projected/ff04a479-3205-4407-ae9e-c9a00a8d8966-kube-api-access-gq4gm\") pod \"ff04a479-3205-4407-ae9e-c9a00a8d8966\" (UID: \"ff04a479-3205-4407-ae9e-c9a00a8d8966\") " Jul 12 18:58:39.492478 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.492159 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ff04a479-3205-4407-ae9e-c9a00a8d8966" (UID: "ff04a479-3205-4407-ae9e-c9a00a8d8966"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:39.492478 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.492283 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ff04a479-3205-4407-ae9e-c9a00a8d8966" (UID: "ff04a479-3205-4407-ae9e-c9a00a8d8966"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:39.492478 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.492313 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-credential-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:39.492478 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.492421 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ff04a479-3205-4407-ae9e-c9a00a8d8966" (UID: "ff04a479-3205-4407-ae9e-c9a00a8d8966"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:39.492718 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.492696 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff04a479-3205-4407-ae9e-c9a00a8d8966-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ff04a479-3205-4407-ae9e-c9a00a8d8966" (UID: "ff04a479-3205-4407-ae9e-c9a00a8d8966"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:58:39.493142 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.493112 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-data" (OuterVolumeSpecName: "istio-data") pod "ff04a479-3205-4407-ae9e-c9a00a8d8966" (UID: "ff04a479-3205-4407-ae9e-c9a00a8d8966"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:39.495535 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.495493 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ff04a479-3205-4407-ae9e-c9a00a8d8966" (UID: "ff04a479-3205-4407-ae9e-c9a00a8d8966"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:58:39.495535 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.495509 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff04a479-3205-4407-ae9e-c9a00a8d8966-kube-api-access-gq4gm" (OuterVolumeSpecName: "kube-api-access-gq4gm") pod "ff04a479-3205-4407-ae9e-c9a00a8d8966" (UID: "ff04a479-3205-4407-ae9e-c9a00a8d8966"). InnerVolumeSpecName "kube-api-access-gq4gm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:58:39.495765 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.495597 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff04a479-3205-4407-ae9e-c9a00a8d8966" (UID: "ff04a479-3205-4407-ae9e-c9a00a8d8966"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:58:39.495765 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.495697 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ff04a479-3205-4407-ae9e-c9a00a8d8966" (UID: "ff04a479-3205-4407-ae9e-c9a00a8d8966"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:39.593020 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.592984 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-envoy\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:39.593020 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.593014 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-workload-certs\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:39.593020 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.593023 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff04a479-3205-4407-ae9e-c9a00a8d8966-istiod-ca-cert\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:39.593278 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.593034 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-workload-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:39.593278 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.593043 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gq4gm\" (UniqueName: \"kubernetes.io/projected/ff04a479-3205-4407-ae9e-c9a00a8d8966-kube-api-access-gq4gm\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:39.593278 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.593053 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-podinfo\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:39.593278 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.593061 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-data\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:39.593278 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:39.593069 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff04a479-3205-4407-ae9e-c9a00a8d8966-istio-token\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:40.299268 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:40.299235 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" Jul 12 18:58:40.299711 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:40.299237 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq" event={"ID":"ff04a479-3205-4407-ae9e-c9a00a8d8966","Type":"ContainerDied","Data":"2a70c8c19849cda819d6c301244931cde5c32490de43a861dd8ad95ea02e2ab5"} Jul 12 18:58:40.299711 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:40.299363 2585 scope.go:117] "RemoveContainer" containerID="79df71b359bc6a7f64995f1f3a14f3ecaae4fd49cb8fb8413d6e369ce2ff1dd4" Jul 12 18:58:40.322028 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:40.321991 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq"] Jul 12 18:58:40.328734 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:40.328699 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-6ee31b-openshift-default-669b897cbc-8c6qq"] Jul 12 18:58:41.306230 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:41.306193 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff04a479-3205-4407-ae9e-c9a00a8d8966" path="/var/lib/kubelet/pods/ff04a479-3205-4407-ae9e-c9a00a8d8966/volumes" Jul 12 18:58:41.537716 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:41.537671 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j"] Jul 12 18:58:41.537977 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:41.537913 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" podUID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerName="istio-proxy" containerID="cri-o://029f1edeb6857d38bd52c0a33edf183c436f0dd0a29da3d7ed99b620ce533fc3" gracePeriod=30 Jul 12 18:58:42.313853 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.313816 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:58:42.357714 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.357671 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl"] Jul 12 18:58:42.358108 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.358093 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff04a479-3205-4407-ae9e-c9a00a8d8966" containerName="istio-proxy" Jul 12 18:58:42.358171 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.358110 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff04a479-3205-4407-ae9e-c9a00a8d8966" containerName="istio-proxy" Jul 12 18:58:42.358208 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.358188 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff04a479-3205-4407-ae9e-c9a00a8d8966" containerName="istio-proxy" Jul 12 18:58:42.363103 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.363077 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.365479 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.365456 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-e0bc69-openshift-default-dockercfg-5tv87\"" Jul 12 18:58:42.372959 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.372915 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl"] Jul 12 18:58:42.419557 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.419522 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47968888-9aac-4986-9983-caf8204877fb-istiod-ca-cert\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.419557 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.419557 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47968888-9aac-4986-9983-caf8204877fb-istio-token\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.419787 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.419588 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzts2\" (UniqueName: \"kubernetes.io/projected/47968888-9aac-4986-9983-caf8204877fb-kube-api-access-rzts2\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.419787 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.419688 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-istio-envoy\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.419787 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.419748 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-credential-socket\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.419787 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.419769 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-workload-socket\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.420017 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.419796 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47968888-9aac-4986-9983-caf8204877fb-istio-podinfo\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.420017 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.419843 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-istio-data\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.420017 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.419923 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-workload-certs\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.521222 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.521123 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47968888-9aac-4986-9983-caf8204877fb-istiod-ca-cert\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.521222 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.521166 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47968888-9aac-4986-9983-caf8204877fb-istio-token\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.521222 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.521184 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzts2\" (UniqueName: \"kubernetes.io/projected/47968888-9aac-4986-9983-caf8204877fb-kube-api-access-rzts2\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.521600 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.521236 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-istio-envoy\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.521600 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.521267 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-credential-socket\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.521600 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.521292 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-workload-socket\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.521600 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.521328 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47968888-9aac-4986-9983-caf8204877fb-istio-podinfo\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.521600 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.521363 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-istio-data\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.521600 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.521420 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-workload-certs\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.521958 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.521893 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-workload-certs\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.522034 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.521956 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-credential-socket\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.522149 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.522128 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47968888-9aac-4986-9983-caf8204877fb-istiod-ca-cert\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.522251 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.522144 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-istio-data\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.522251 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.522154 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-workload-socket\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.523902 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.523881 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-istio-envoy\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.524066 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.524047 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47968888-9aac-4986-9983-caf8204877fb-istio-podinfo\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.529034 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.528992 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47968888-9aac-4986-9983-caf8204877fb-istio-token\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.529157 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.529110 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzts2\" (UniqueName: \"kubernetes.io/projected/47968888-9aac-4986-9983-caf8204877fb-kube-api-access-rzts2\") pod \"e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.675409 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.675301 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:42.821467 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.821376 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl"] Jul 12 18:58:42.823828 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:58:42.823793 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47968888_9aac_4986_9983_caf8204877fb.slice/crio-30a1300ee29e3ca78bff927684a6a366d10a51f25500a37b37bd560bf66936a9 WatchSource:0}: Error finding container 30a1300ee29e3ca78bff927684a6a366d10a51f25500a37b37bd560bf66936a9: Status 404 returned error can't find the container with id 30a1300ee29e3ca78bff927684a6a366d10a51f25500a37b37bd560bf66936a9 Jul 12 18:58:42.826198 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.826160 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:42.826260 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.826233 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:42.826294 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:42.826260 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:43.319692 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:43.319652 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" event={"ID":"47968888-9aac-4986-9983-caf8204877fb","Type":"ContainerStarted","Data":"01b5037cbcd900a2107f61a936c2e63bf53e393eed8a1abb00c36b9861f82db7"} Jul 12 18:58:43.320103 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:43.319711 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" event={"ID":"47968888-9aac-4986-9983-caf8204877fb","Type":"ContainerStarted","Data":"30a1300ee29e3ca78bff927684a6a366d10a51f25500a37b37bd560bf66936a9"} Jul 12 18:58:43.340766 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:43.340715 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" podStartSLOduration=1.340697398 podStartE2EDuration="1.340697398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:58:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:58:43.337839164 +0000 UTC m=+2014.537306194" watchObservedRunningTime="2026-07-12 18:58:43.340697398 +0000 UTC m=+2014.540164427" Jul 12 18:58:43.676178 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:43.676082 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:43.681478 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:43.681445 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:44.325010 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:44.324971 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:44.326037 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:44.326014 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:58:46.802414 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.802341 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:46.865253 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.865212 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-workload-socket\") pod \"780038ad-a845-4003-95f0-e5e36fcec5d8\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " Jul 12 18:58:46.865253 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.865253 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-token\") pod \"780038ad-a845-4003-95f0-e5e36fcec5d8\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " Jul 12 18:58:46.865512 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.865275 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780038ad-a845-4003-95f0-e5e36fcec5d8-istiod-ca-cert\") pod \"780038ad-a845-4003-95f0-e5e36fcec5d8\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " Jul 12 18:58:46.865512 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.865292 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-envoy\") pod \"780038ad-a845-4003-95f0-e5e36fcec5d8\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " Jul 12 18:58:46.865512 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.865317 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-workload-certs\") pod \"780038ad-a845-4003-95f0-e5e36fcec5d8\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " Jul 12 18:58:46.865512 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.865397 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-podinfo\") pod \"780038ad-a845-4003-95f0-e5e36fcec5d8\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " Jul 12 18:58:46.865512 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.865474 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-data\") pod \"780038ad-a845-4003-95f0-e5e36fcec5d8\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " Jul 12 18:58:46.865791 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.865523 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "780038ad-a845-4003-95f0-e5e36fcec5d8" (UID: "780038ad-a845-4003-95f0-e5e36fcec5d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:46.865791 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.865536 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-credential-socket\") pod \"780038ad-a845-4003-95f0-e5e36fcec5d8\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " Jul 12 18:58:46.865791 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.865638 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z54h5\" (UniqueName: \"kubernetes.io/projected/780038ad-a845-4003-95f0-e5e36fcec5d8-kube-api-access-z54h5\") pod \"780038ad-a845-4003-95f0-e5e36fcec5d8\" (UID: \"780038ad-a845-4003-95f0-e5e36fcec5d8\") " Jul 12 18:58:46.865791 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.865756 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "780038ad-a845-4003-95f0-e5e36fcec5d8" (UID: "780038ad-a845-4003-95f0-e5e36fcec5d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:46.866033 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.865922 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "780038ad-a845-4003-95f0-e5e36fcec5d8" (UID: "780038ad-a845-4003-95f0-e5e36fcec5d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:46.866033 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.865759 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/780038ad-a845-4003-95f0-e5e36fcec5d8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "780038ad-a845-4003-95f0-e5e36fcec5d8" (UID: "780038ad-a845-4003-95f0-e5e36fcec5d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:58:46.866142 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.866051 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-credential-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:46.866142 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.866072 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-workload-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:46.866142 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.866086 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780038ad-a845-4003-95f0-e5e36fcec5d8-istiod-ca-cert\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:46.866142 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.866099 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-workload-certs\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:46.866645 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.866618 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-data" (OuterVolumeSpecName: "istio-data") pod "780038ad-a845-4003-95f0-e5e36fcec5d8" (UID: "780038ad-a845-4003-95f0-e5e36fcec5d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:46.867651 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.867625 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "780038ad-a845-4003-95f0-e5e36fcec5d8" (UID: "780038ad-a845-4003-95f0-e5e36fcec5d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:58:46.867651 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.867644 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-token" (OuterVolumeSpecName: "istio-token") pod "780038ad-a845-4003-95f0-e5e36fcec5d8" (UID: "780038ad-a845-4003-95f0-e5e36fcec5d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:58:46.867779 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.867674 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "780038ad-a845-4003-95f0-e5e36fcec5d8" (UID: "780038ad-a845-4003-95f0-e5e36fcec5d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:58:46.867916 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.867900 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/780038ad-a845-4003-95f0-e5e36fcec5d8-kube-api-access-z54h5" (OuterVolumeSpecName: "kube-api-access-z54h5") pod "780038ad-a845-4003-95f0-e5e36fcec5d8" (UID: "780038ad-a845-4003-95f0-e5e36fcec5d8"). InnerVolumeSpecName "kube-api-access-z54h5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:58:46.966925 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.966886 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z54h5\" (UniqueName: \"kubernetes.io/projected/780038ad-a845-4003-95f0-e5e36fcec5d8-kube-api-access-z54h5\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:46.966925 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.966920 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-token\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:46.967171 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.966960 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-envoy\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:46.967171 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.966971 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-podinfo\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:46.967171 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:46.966981 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780038ad-a845-4003-95f0-e5e36fcec5d8-istio-data\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 18:58:47.344529 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:47.344493 2585 generic.go:356] "Generic (PLEG): container finished" podID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerID="029f1edeb6857d38bd52c0a33edf183c436f0dd0a29da3d7ed99b620ce533fc3" exitCode=0 Jul 12 18:58:47.344725 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:47.344552 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" Jul 12 18:58:47.344725 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:47.344581 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" event={"ID":"780038ad-a845-4003-95f0-e5e36fcec5d8","Type":"ContainerDied","Data":"029f1edeb6857d38bd52c0a33edf183c436f0dd0a29da3d7ed99b620ce533fc3"} Jul 12 18:58:47.344725 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:47.344637 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j" event={"ID":"780038ad-a845-4003-95f0-e5e36fcec5d8","Type":"ContainerDied","Data":"e201187385b2a8e5d993e4aad9ef173dd1465b71c14f2770d7ba53f2b70c3699"} Jul 12 18:58:47.344725 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:47.344662 2585 scope.go:117] "RemoveContainer" containerID="029f1edeb6857d38bd52c0a33edf183c436f0dd0a29da3d7ed99b620ce533fc3" Jul 12 18:58:47.354296 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:47.354275 2585 scope.go:117] "RemoveContainer" containerID="85bbc825012109b177568c35cb41e705e6a4b265089448d5e521f3089151ee00" Jul 12 18:58:47.363475 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:47.363448 2585 scope.go:117] "RemoveContainer" containerID="029f1edeb6857d38bd52c0a33edf183c436f0dd0a29da3d7ed99b620ce533fc3" Jul 12 18:58:47.364059 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:47.363845 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j"] Jul 12 18:58:47.364059 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:58:47.363848 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"029f1edeb6857d38bd52c0a33edf183c436f0dd0a29da3d7ed99b620ce533fc3\": container with ID starting with 029f1edeb6857d38bd52c0a33edf183c436f0dd0a29da3d7ed99b620ce533fc3 not found: ID does not exist" containerID="029f1edeb6857d38bd52c0a33edf183c436f0dd0a29da3d7ed99b620ce533fc3" Jul 12 18:58:47.364059 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:47.363939 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"029f1edeb6857d38bd52c0a33edf183c436f0dd0a29da3d7ed99b620ce533fc3"} err="failed to get container status \"029f1edeb6857d38bd52c0a33edf183c436f0dd0a29da3d7ed99b620ce533fc3\": rpc error: code = NotFound desc = could not find container \"029f1edeb6857d38bd52c0a33edf183c436f0dd0a29da3d7ed99b620ce533fc3\": container with ID starting with 029f1edeb6857d38bd52c0a33edf183c436f0dd0a29da3d7ed99b620ce533fc3 not found: ID does not exist" Jul 12 18:58:47.364059 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:47.363967 2585 scope.go:117] "RemoveContainer" containerID="85bbc825012109b177568c35cb41e705e6a4b265089448d5e521f3089151ee00" Jul 12 18:58:47.364757 ip-10-0-140-170 kubenswrapper[2585]: E0712 18:58:47.364731 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85bbc825012109b177568c35cb41e705e6a4b265089448d5e521f3089151ee00\": container with ID starting with 85bbc825012109b177568c35cb41e705e6a4b265089448d5e521f3089151ee00 not found: ID does not exist" containerID="85bbc825012109b177568c35cb41e705e6a4b265089448d5e521f3089151ee00" Jul 12 18:58:47.364811 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:47.364768 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85bbc825012109b177568c35cb41e705e6a4b265089448d5e521f3089151ee00"} err="failed to get container status \"85bbc825012109b177568c35cb41e705e6a4b265089448d5e521f3089151ee00\": rpc error: code = NotFound desc = could not find container \"85bbc825012109b177568c35cb41e705e6a4b265089448d5e521f3089151ee00\": container with ID starting with 85bbc825012109b177568c35cb41e705e6a4b265089448d5e521f3089151ee00 not found: ID does not exist" Jul 12 18:58:47.366914 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:47.366889 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-498f3f-openshift-default-7764c5b5cc-lr82j"] Jul 12 18:58:49.310613 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:49.310569 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="780038ad-a845-4003-95f0-e5e36fcec5d8" path="/var/lib/kubelet/pods/780038ad-a845-4003-95f0-e5e36fcec5d8/volumes" Jul 12 18:58:50.282260 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.282214 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs"] Jul 12 18:58:50.282831 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.282811 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff04a479-3205-4407-ae9e-c9a00a8d8966" containerName="istio-proxy" Jul 12 18:58:50.282890 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.282836 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff04a479-3205-4407-ae9e-c9a00a8d8966" containerName="istio-proxy" Jul 12 18:58:50.282890 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.282850 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerName="istio-proxy" Jul 12 18:58:50.282890 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.282859 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerName="istio-proxy" Jul 12 18:58:50.282890 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.282880 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerName="istio-proxy" Jul 12 18:58:50.283032 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.282890 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerName="istio-proxy" Jul 12 18:58:50.283032 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.283011 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerName="istio-proxy" Jul 12 18:58:50.283032 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.283026 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="780038ad-a845-4003-95f0-e5e36fcec5d8" containerName="istio-proxy" Jul 12 18:58:50.283125 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.283036 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff04a479-3205-4407-ae9e-c9a00a8d8966" containerName="istio-proxy" Jul 12 18:58:50.288047 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.288024 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.290266 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.290243 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-92be65-openshift-default-dockercfg-wds58\"" Jul 12 18:58:50.295772 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.295749 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs"] Jul 12 18:58:50.397543 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.397502 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-data\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.398299 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.398274 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-workload-certs\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.398748 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.398726 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-workload-socket\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.398958 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.398922 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-token\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.399165 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.399134 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xvbp\" (UniqueName: \"kubernetes.io/projected/ba03b06e-c152-4e85-b78a-66473fb087ea-kube-api-access-4xvbp\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.399286 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.399214 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-credential-socket\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.399286 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.399267 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-podinfo\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.399453 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.399412 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba03b06e-c152-4e85-b78a-66473fb087ea-istiod-ca-cert\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.399555 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.399531 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-envoy\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.500407 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.500373 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-credential-socket\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.500677 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.500416 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-podinfo\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.500677 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.500443 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba03b06e-c152-4e85-b78a-66473fb087ea-istiod-ca-cert\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.500677 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.500468 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-envoy\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.500677 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.500486 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-data\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.500677 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.500509 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-workload-certs\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.500677 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.500528 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-workload-socket\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.500677 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.500552 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-token\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.500677 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.500653 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xvbp\" (UniqueName: \"kubernetes.io/projected/ba03b06e-c152-4e85-b78a-66473fb087ea-kube-api-access-4xvbp\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.501159 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.500856 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-credential-socket\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.501159 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.501039 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-workload-certs\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.501159 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.501087 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-workload-socket\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.501281 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.501221 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-data\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.501321 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.501281 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba03b06e-c152-4e85-b78a-66473fb087ea-istiod-ca-cert\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.502983 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.502961 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-envoy\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.503158 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.503139 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-podinfo\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.508114 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.508094 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-token\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.508327 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.508306 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xvbp\" (UniqueName: \"kubernetes.io/projected/ba03b06e-c152-4e85-b78a-66473fb087ea-kube-api-access-4xvbp\") pod \"e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.599245 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.599144 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:50.749334 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.749300 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs"] Jul 12 18:58:50.750643 ip-10-0-140-170 kubenswrapper[2585]: W0712 18:58:50.750613 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba03b06e_c152_4e85_b78a_66473fb087ea.slice/crio-c6e53f62fdade1fb3e4d6a71578bf337e4e661fab3bd3e9499f94c7aab2bd089 WatchSource:0}: Error finding container c6e53f62fdade1fb3e4d6a71578bf337e4e661fab3bd3e9499f94c7aab2bd089: Status 404 returned error can't find the container with id c6e53f62fdade1fb3e4d6a71578bf337e4e661fab3bd3e9499f94c7aab2bd089 Jul 12 18:58:50.752824 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.752791 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:50.752910 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.752862 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:50.752910 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.752894 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:50.815314 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:50.815285 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:58:51.369702 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:51.369658 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" event={"ID":"ba03b06e-c152-4e85-b78a-66473fb087ea","Type":"ContainerStarted","Data":"26234f199d7b217f9113f4b0a6f97fdca71a98eba74149fe01e15242165c6646"} Jul 12 18:58:51.369916 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:51.369714 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" event={"ID":"ba03b06e-c152-4e85-b78a-66473fb087ea","Type":"ContainerStarted","Data":"c6e53f62fdade1fb3e4d6a71578bf337e4e661fab3bd3e9499f94c7aab2bd089"} Jul 12 18:58:51.388993 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:51.388924 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" podStartSLOduration=1.388903934 podStartE2EDuration="1.388903934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:58:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:58:51.387923718 +0000 UTC m=+2022.587390745" watchObservedRunningTime="2026-07-12 18:58:51.388903934 +0000 UTC m=+2022.588370965" Jul 12 18:58:51.600127 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:51.600086 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:51.605118 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:51.605089 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:52.375539 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:52.375507 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:52.376504 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:52.376482 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:58:59.210072 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:59.210032 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 18:58:59.326678 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:59.326640 2585 patch_prober.go:28] interesting pod/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 12 18:58:59.326851 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:59.326713 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" podUID="47968888-9aac-4986-9983-caf8204877fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 12 18:58:59.414158 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:59.414123 2585 generic.go:356] "Generic (PLEG): container finished" podID="47968888-9aac-4986-9983-caf8204877fb" containerID="01b5037cbcd900a2107f61a936c2e63bf53e393eed8a1abb00c36b9861f82db7" exitCode=0 Jul 12 18:58:59.414338 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:59.414201 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" event={"ID":"47968888-9aac-4986-9983-caf8204877fb","Type":"ContainerDied","Data":"01b5037cbcd900a2107f61a936c2e63bf53e393eed8a1abb00c36b9861f82db7"} Jul 12 18:58:59.414614 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:59.414600 2585 scope.go:117] "RemoveContainer" containerID="01b5037cbcd900a2107f61a936c2e63bf53e393eed8a1abb00c36b9861f82db7" Jul 12 18:58:59.415689 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:59.415658 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:59.415773 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:59.415736 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:58:59.415773 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:58:59.415767 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:59:00.421554 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:00.421516 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" event={"ID":"47968888-9aac-4986-9983-caf8204877fb","Type":"ContainerStarted","Data":"1b475eb0acd8325457273af75ced2bef2b6ae1becdba326f3a5712ebe26de4bc"} Jul 12 18:59:00.676100 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:00.675999 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:59:00.676298 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:00.676172 2585 patch_prober.go:28] interesting pod/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 12 18:59:00.676298 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:00.676212 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" podUID="47968888-9aac-4986-9983-caf8204877fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 12 18:59:01.676383 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:01.676335 2585 patch_prober.go:28] interesting pod/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 12 18:59:01.676809 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:01.676433 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" podUID="47968888-9aac-4986-9983-caf8204877fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 12 18:59:02.676641 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:02.676597 2585 patch_prober.go:28] interesting pod/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 12 18:59:02.677351 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:02.676691 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" podUID="47968888-9aac-4986-9983-caf8204877fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 12 18:59:03.676256 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:03.676213 2585 patch_prober.go:28] interesting pod/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 12 18:59:03.676448 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:03.676285 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" podUID="47968888-9aac-4986-9983-caf8204877fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 12 18:59:04.681145 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:04.681113 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:59:04.681623 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:04.681388 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:59:04.682151 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:04.682129 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 18:59:07.554741 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:07.554088 2585 patch_prober.go:28] interesting pod/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": read tcp 10.132.0.2:56904->10.132.0.58:15021: read: connection reset by peer" start-of-body= Jul 12 18:59:07.554741 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:07.554185 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" podUID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": read tcp 10.132.0.2:56904->10.132.0.58:15021: read: connection reset by peer" Jul 12 18:59:08.490205 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:08.490171 2585 generic.go:356] "Generic (PLEG): container finished" podID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerID="26234f199d7b217f9113f4b0a6f97fdca71a98eba74149fe01e15242165c6646" exitCode=0 Jul 12 18:59:08.490419 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:08.490244 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" event={"ID":"ba03b06e-c152-4e85-b78a-66473fb087ea","Type":"ContainerDied","Data":"26234f199d7b217f9113f4b0a6f97fdca71a98eba74149fe01e15242165c6646"} Jul 12 18:59:08.490724 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:08.490707 2585 scope.go:117] "RemoveContainer" containerID="26234f199d7b217f9113f4b0a6f97fdca71a98eba74149fe01e15242165c6646" Jul 12 18:59:08.491799 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:08.491773 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:59:08.491874 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:08.491854 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:59:08.491913 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:08.491893 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 12 18:59:09.499419 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:09.499384 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" event={"ID":"ba03b06e-c152-4e85-b78a-66473fb087ea","Type":"ContainerStarted","Data":"d2261eae3e8794ca0fd0c4f045951a2c218999b8334225c2f0461bf192500f2f"} Jul 12 18:59:09.599337 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:09.599302 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:59:09.599577 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:09.599553 2585 patch_prober.go:28] interesting pod/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 12 18:59:09.599627 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:09.599605 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" podUID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 12 18:59:10.600046 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:10.600009 2585 patch_prober.go:28] interesting pod/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 12 18:59:10.600454 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:10.600086 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" podUID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 12 18:59:11.600318 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:11.600277 2585 patch_prober.go:28] interesting pod/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 12 18:59:11.600712 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:11.600352 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" podUID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 12 18:59:12.599704 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:12.599666 2585 patch_prober.go:28] interesting pod/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 12 18:59:12.599887 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:12.599737 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" podUID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 12 18:59:13.599827 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:13.599791 2585 patch_prober.go:28] interesting pod/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 12 18:59:13.600270 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:13.599848 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" podUID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 12 18:59:14.603603 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:14.603564 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:59:14.604116 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:14.603964 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 18:59:14.604587 ip-10-0-140-170 kubenswrapper[2585]: I0712 18:59:14.604568 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 19:00:09.474754 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:00:09.474724 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 19:00:09.479654 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:00:09.479630 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 19:02:05.514706 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:05.514664 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 19:02:08.853452 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:08.853416 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl"] Jul 12 19:02:08.853979 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:08.853647 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" podUID="47968888-9aac-4986-9983-caf8204877fb" containerName="istio-proxy" containerID="cri-o://1b475eb0acd8325457273af75ced2bef2b6ae1becdba326f3a5712ebe26de4bc" gracePeriod=30 Jul 12 19:02:14.102333 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.102293 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-45x44"] Jul 12 19:02:14.126434 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.126360 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 19:02:14.208595 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.208547 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47968888-9aac-4986-9983-caf8204877fb-istiod-ca-cert\") pod \"47968888-9aac-4986-9983-caf8204877fb\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " Jul 12 19:02:14.208595 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.208594 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-istio-data\") pod \"47968888-9aac-4986-9983-caf8204877fb\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " Jul 12 19:02:14.208848 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.208624 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-istio-envoy\") pod \"47968888-9aac-4986-9983-caf8204877fb\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " Jul 12 19:02:14.208848 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.208650 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-credential-socket\") pod \"47968888-9aac-4986-9983-caf8204877fb\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " Jul 12 19:02:14.208848 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.208687 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzts2\" (UniqueName: \"kubernetes.io/projected/47968888-9aac-4986-9983-caf8204877fb-kube-api-access-rzts2\") pod \"47968888-9aac-4986-9983-caf8204877fb\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " Jul 12 19:02:14.208848 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.208723 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-workload-socket\") pod \"47968888-9aac-4986-9983-caf8204877fb\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " Jul 12 19:02:14.208848 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.208748 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-workload-certs\") pod \"47968888-9aac-4986-9983-caf8204877fb\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " Jul 12 19:02:14.208848 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.208834 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47968888-9aac-4986-9983-caf8204877fb-istio-token\") pod \"47968888-9aac-4986-9983-caf8204877fb\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " Jul 12 19:02:14.209208 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.208861 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47968888-9aac-4986-9983-caf8204877fb-istio-podinfo\") pod \"47968888-9aac-4986-9983-caf8204877fb\" (UID: \"47968888-9aac-4986-9983-caf8204877fb\") " Jul 12 19:02:14.209208 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.209130 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47968888-9aac-4986-9983-caf8204877fb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "47968888-9aac-4986-9983-caf8204877fb" (UID: "47968888-9aac-4986-9983-caf8204877fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 19:02:14.209208 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.209152 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "47968888-9aac-4986-9983-caf8204877fb" (UID: "47968888-9aac-4986-9983-caf8204877fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:02:14.209367 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.209275 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "47968888-9aac-4986-9983-caf8204877fb" (UID: "47968888-9aac-4986-9983-caf8204877fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:02:14.209423 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.209388 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47968888-9aac-4986-9983-caf8204877fb-istiod-ca-cert\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:14.209423 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.209408 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-credential-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:14.209521 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.209422 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-workload-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:14.209521 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.209462 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "47968888-9aac-4986-9983-caf8204877fb" (UID: "47968888-9aac-4986-9983-caf8204877fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:02:14.209618 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.209586 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-istio-data" (OuterVolumeSpecName: "istio-data") pod "47968888-9aac-4986-9983-caf8204877fb" (UID: "47968888-9aac-4986-9983-caf8204877fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:02:14.211267 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.211239 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47968888-9aac-4986-9983-caf8204877fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "47968888-9aac-4986-9983-caf8204877fb" (UID: "47968888-9aac-4986-9983-caf8204877fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 19:02:14.211267 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.211256 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47968888-9aac-4986-9983-caf8204877fb-kube-api-access-rzts2" (OuterVolumeSpecName: "kube-api-access-rzts2") pod "47968888-9aac-4986-9983-caf8204877fb" (UID: "47968888-9aac-4986-9983-caf8204877fb"). InnerVolumeSpecName "kube-api-access-rzts2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 19:02:14.211441 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.211425 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "47968888-9aac-4986-9983-caf8204877fb" (UID: "47968888-9aac-4986-9983-caf8204877fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:02:14.211482 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.211461 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/47968888-9aac-4986-9983-caf8204877fb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "47968888-9aac-4986-9983-caf8204877fb" (UID: "47968888-9aac-4986-9983-caf8204877fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 19:02:14.310722 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.310677 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47968888-9aac-4986-9983-caf8204877fb-istio-token\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:14.310722 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.310714 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47968888-9aac-4986-9983-caf8204877fb-istio-podinfo\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:14.310722 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.310727 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-istio-data\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:14.311047 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.310738 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-istio-envoy\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:14.311047 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.310750 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzts2\" (UniqueName: \"kubernetes.io/projected/47968888-9aac-4986-9983-caf8204877fb-kube-api-access-rzts2\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:14.311047 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.310762 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47968888-9aac-4986-9983-caf8204877fb-workload-certs\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:14.549054 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.549014 2585 generic.go:356] "Generic (PLEG): container finished" podID="47968888-9aac-4986-9983-caf8204877fb" containerID="1b475eb0acd8325457273af75ced2bef2b6ae1becdba326f3a5712ebe26de4bc" exitCode=0 Jul 12 19:02:14.549244 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.549074 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" Jul 12 19:02:14.549244 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.549078 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" event={"ID":"47968888-9aac-4986-9983-caf8204877fb","Type":"ContainerDied","Data":"1b475eb0acd8325457273af75ced2bef2b6ae1becdba326f3a5712ebe26de4bc"} Jul 12 19:02:14.549244 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.549132 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl" event={"ID":"47968888-9aac-4986-9983-caf8204877fb","Type":"ContainerDied","Data":"30a1300ee29e3ca78bff927684a6a366d10a51f25500a37b37bd560bf66936a9"} Jul 12 19:02:14.549244 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.549162 2585 scope.go:117] "RemoveContainer" containerID="1b475eb0acd8325457273af75ced2bef2b6ae1becdba326f3a5712ebe26de4bc" Jul 12 19:02:14.558990 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.558969 2585 scope.go:117] "RemoveContainer" containerID="01b5037cbcd900a2107f61a936c2e63bf53e393eed8a1abb00c36b9861f82db7" Jul 12 19:02:14.568602 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.568539 2585 scope.go:117] "RemoveContainer" containerID="1b475eb0acd8325457273af75ced2bef2b6ae1becdba326f3a5712ebe26de4bc" Jul 12 19:02:14.568969 ip-10-0-140-170 kubenswrapper[2585]: E0712 19:02:14.568946 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b475eb0acd8325457273af75ced2bef2b6ae1becdba326f3a5712ebe26de4bc\": container with ID starting with 1b475eb0acd8325457273af75ced2bef2b6ae1becdba326f3a5712ebe26de4bc not found: ID does not exist" containerID="1b475eb0acd8325457273af75ced2bef2b6ae1becdba326f3a5712ebe26de4bc" Jul 12 19:02:14.569080 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.568982 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b475eb0acd8325457273af75ced2bef2b6ae1becdba326f3a5712ebe26de4bc"} err="failed to get container status \"1b475eb0acd8325457273af75ced2bef2b6ae1becdba326f3a5712ebe26de4bc\": rpc error: code = NotFound desc = could not find container \"1b475eb0acd8325457273af75ced2bef2b6ae1becdba326f3a5712ebe26de4bc\": container with ID starting with 1b475eb0acd8325457273af75ced2bef2b6ae1becdba326f3a5712ebe26de4bc not found: ID does not exist" Jul 12 19:02:14.569080 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.569009 2585 scope.go:117] "RemoveContainer" containerID="01b5037cbcd900a2107f61a936c2e63bf53e393eed8a1abb00c36b9861f82db7" Jul 12 19:02:14.569322 ip-10-0-140-170 kubenswrapper[2585]: E0712 19:02:14.569305 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01b5037cbcd900a2107f61a936c2e63bf53e393eed8a1abb00c36b9861f82db7\": container with ID starting with 01b5037cbcd900a2107f61a936c2e63bf53e393eed8a1abb00c36b9861f82db7 not found: ID does not exist" containerID="01b5037cbcd900a2107f61a936c2e63bf53e393eed8a1abb00c36b9861f82db7" Jul 12 19:02:14.569388 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.569329 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01b5037cbcd900a2107f61a936c2e63bf53e393eed8a1abb00c36b9861f82db7"} err="failed to get container status \"01b5037cbcd900a2107f61a936c2e63bf53e393eed8a1abb00c36b9861f82db7\": rpc error: code = NotFound desc = could not find container \"01b5037cbcd900a2107f61a936c2e63bf53e393eed8a1abb00c36b9861f82db7\": container with ID starting with 01b5037cbcd900a2107f61a936c2e63bf53e393eed8a1abb00c36b9861f82db7 not found: ID does not exist" Jul 12 19:02:14.571399 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.571377 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl"] Jul 12 19:02:14.575639 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:14.575610 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-e0bc69-openshift-default-5d6dd88479-97fwl"] Jul 12 19:02:15.307104 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:15.307069 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47968888-9aac-4986-9983-caf8204877fb" path="/var/lib/kubelet/pods/47968888-9aac-4986-9983-caf8204877fb/volumes" Jul 12 19:02:15.596604 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:15.596509 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs"] Jul 12 19:02:15.596796 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:15.596747 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" podUID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerName="istio-proxy" containerID="cri-o://d2261eae3e8794ca0fd0c4f045951a2c218999b8334225c2f0461bf192500f2f" gracePeriod=30 Jul 12 19:02:20.862382 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.862347 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 19:02:20.971890 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.971784 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-workload-socket\") pod \"ba03b06e-c152-4e85-b78a-66473fb087ea\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " Jul 12 19:02:20.971890 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.971828 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-podinfo\") pod \"ba03b06e-c152-4e85-b78a-66473fb087ea\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " Jul 12 19:02:20.972169 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.971918 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-token\") pod \"ba03b06e-c152-4e85-b78a-66473fb087ea\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " Jul 12 19:02:20.972169 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.972051 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-data\") pod \"ba03b06e-c152-4e85-b78a-66473fb087ea\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " Jul 12 19:02:20.972169 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.972070 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-workload-certs\") pod \"ba03b06e-c152-4e85-b78a-66473fb087ea\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " Jul 12 19:02:20.972169 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.972091 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-credential-socket\") pod \"ba03b06e-c152-4e85-b78a-66473fb087ea\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " Jul 12 19:02:20.972169 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.972117 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-envoy\") pod \"ba03b06e-c152-4e85-b78a-66473fb087ea\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " Jul 12 19:02:20.972169 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.972136 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba03b06e-c152-4e85-b78a-66473fb087ea-istiod-ca-cert\") pod \"ba03b06e-c152-4e85-b78a-66473fb087ea\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " Jul 12 19:02:20.972169 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.972166 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xvbp\" (UniqueName: \"kubernetes.io/projected/ba03b06e-c152-4e85-b78a-66473fb087ea-kube-api-access-4xvbp\") pod \"ba03b06e-c152-4e85-b78a-66473fb087ea\" (UID: \"ba03b06e-c152-4e85-b78a-66473fb087ea\") " Jul 12 19:02:20.972536 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.972214 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba03b06e-c152-4e85-b78a-66473fb087ea" (UID: "ba03b06e-c152-4e85-b78a-66473fb087ea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:02:20.972536 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.972335 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba03b06e-c152-4e85-b78a-66473fb087ea" (UID: "ba03b06e-c152-4e85-b78a-66473fb087ea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:02:20.972536 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.972500 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-workload-certs\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:20.972536 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.972520 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-workload-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:20.972759 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.972602 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba03b06e-c152-4e85-b78a-66473fb087ea" (UID: "ba03b06e-c152-4e85-b78a-66473fb087ea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:02:20.972759 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.972644 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba03b06e-c152-4e85-b78a-66473fb087ea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba03b06e-c152-4e85-b78a-66473fb087ea" (UID: "ba03b06e-c152-4e85-b78a-66473fb087ea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 19:02:20.973178 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.973146 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba03b06e-c152-4e85-b78a-66473fb087ea" (UID: "ba03b06e-c152-4e85-b78a-66473fb087ea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:02:20.974388 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.974362 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba03b06e-c152-4e85-b78a-66473fb087ea" (UID: "ba03b06e-c152-4e85-b78a-66473fb087ea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:02:20.974511 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.974422 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba03b06e-c152-4e85-b78a-66473fb087ea" (UID: "ba03b06e-c152-4e85-b78a-66473fb087ea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 19:02:20.974639 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.974617 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba03b06e-c152-4e85-b78a-66473fb087ea" (UID: "ba03b06e-c152-4e85-b78a-66473fb087ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 19:02:20.974693 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:20.974651 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba03b06e-c152-4e85-b78a-66473fb087ea-kube-api-access-4xvbp" (OuterVolumeSpecName: "kube-api-access-4xvbp") pod "ba03b06e-c152-4e85-b78a-66473fb087ea" (UID: "ba03b06e-c152-4e85-b78a-66473fb087ea"). InnerVolumeSpecName "kube-api-access-4xvbp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 19:02:21.073751 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.073707 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-podinfo\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:21.073751 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.073744 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-token\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:21.073751 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.073756 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-data\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:21.074067 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.073767 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-credential-socket\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:21.074067 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.073780 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba03b06e-c152-4e85-b78a-66473fb087ea-istio-envoy\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:21.074067 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.073791 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba03b06e-c152-4e85-b78a-66473fb087ea-istiod-ca-cert\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:21.074067 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.073802 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xvbp\" (UniqueName: \"kubernetes.io/projected/ba03b06e-c152-4e85-b78a-66473fb087ea-kube-api-access-4xvbp\") on node \"ip-10-0-140-170.ec2.internal\" DevicePath \"\"" Jul 12 19:02:21.176688 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.176652 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-l5nxv_b96066fe-ee3b-40a9-80fb-e1fd4bbeac68/server/0.log" Jul 12 19:02:21.288574 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.288541 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6f7h9_59e7df2c-c8b5-4163-9bc7-baa9125014df/manager/0.log" Jul 12 19:02:21.515913 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.515885 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f88dd4c8-l6j59_3d6039e0-174a-4524-ba46-500c1604c808/copy-manifests/0.log" Jul 12 19:02:21.522627 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.522604 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f88dd4c8-l6j59_3d6039e0-174a-4524-ba46-500c1604c808/manager/0.log" Jul 12 19:02:21.592673 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.592568 2585 generic.go:356] "Generic (PLEG): container finished" podID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerID="d2261eae3e8794ca0fd0c4f045951a2c218999b8334225c2f0461bf192500f2f" exitCode=0 Jul 12 19:02:21.592673 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.592618 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" Jul 12 19:02:21.592673 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.592649 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" event={"ID":"ba03b06e-c152-4e85-b78a-66473fb087ea","Type":"ContainerDied","Data":"d2261eae3e8794ca0fd0c4f045951a2c218999b8334225c2f0461bf192500f2f"} Jul 12 19:02:21.592956 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.592706 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs" event={"ID":"ba03b06e-c152-4e85-b78a-66473fb087ea","Type":"ContainerDied","Data":"c6e53f62fdade1fb3e4d6a71578bf337e4e661fab3bd3e9499f94c7aab2bd089"} Jul 12 19:02:21.592956 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.592729 2585 scope.go:117] "RemoveContainer" containerID="d2261eae3e8794ca0fd0c4f045951a2c218999b8334225c2f0461bf192500f2f" Jul 12 19:02:21.602746 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.602727 2585 scope.go:117] "RemoveContainer" containerID="26234f199d7b217f9113f4b0a6f97fdca71a98eba74149fe01e15242165c6646" Jul 12 19:02:21.610085 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.610055 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs"] Jul 12 19:02:21.612908 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.612887 2585 scope.go:117] "RemoveContainer" containerID="d2261eae3e8794ca0fd0c4f045951a2c218999b8334225c2f0461bf192500f2f" Jul 12 19:02:21.613243 ip-10-0-140-170 kubenswrapper[2585]: E0712 19:02:21.613225 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2261eae3e8794ca0fd0c4f045951a2c218999b8334225c2f0461bf192500f2f\": container with ID starting with d2261eae3e8794ca0fd0c4f045951a2c218999b8334225c2f0461bf192500f2f not found: ID does not exist" containerID="d2261eae3e8794ca0fd0c4f045951a2c218999b8334225c2f0461bf192500f2f" Jul 12 19:02:21.613301 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.613254 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2261eae3e8794ca0fd0c4f045951a2c218999b8334225c2f0461bf192500f2f"} err="failed to get container status \"d2261eae3e8794ca0fd0c4f045951a2c218999b8334225c2f0461bf192500f2f\": rpc error: code = NotFound desc = could not find container \"d2261eae3e8794ca0fd0c4f045951a2c218999b8334225c2f0461bf192500f2f\": container with ID starting with d2261eae3e8794ca0fd0c4f045951a2c218999b8334225c2f0461bf192500f2f not found: ID does not exist" Jul 12 19:02:21.613301 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.613274 2585 scope.go:117] "RemoveContainer" containerID="26234f199d7b217f9113f4b0a6f97fdca71a98eba74149fe01e15242165c6646" Jul 12 19:02:21.613575 ip-10-0-140-170 kubenswrapper[2585]: E0712 19:02:21.613558 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26234f199d7b217f9113f4b0a6f97fdca71a98eba74149fe01e15242165c6646\": container with ID starting with 26234f199d7b217f9113f4b0a6f97fdca71a98eba74149fe01e15242165c6646 not found: ID does not exist" containerID="26234f199d7b217f9113f4b0a6f97fdca71a98eba74149fe01e15242165c6646" Jul 12 19:02:21.613632 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.613578 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26234f199d7b217f9113f4b0a6f97fdca71a98eba74149fe01e15242165c6646"} err="failed to get container status \"26234f199d7b217f9113f4b0a6f97fdca71a98eba74149fe01e15242165c6646\": rpc error: code = NotFound desc = could not find container \"26234f199d7b217f9113f4b0a6f97fdca71a98eba74149fe01e15242165c6646\": container with ID starting with 26234f199d7b217f9113f4b0a6f97fdca71a98eba74149fe01e15242165c6646 not found: ID does not exist" Jul 12 19:02:21.614052 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:21.614034 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-92be65-openshift-default-79bd8c6ccf-x7hfs"] Jul 12 19:02:22.373475 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:22.373443 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx_f3e22a16-2734-45fb-9e1f-cfba0060dfa7/util/0.log" Jul 12 19:02:22.379740 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:22.379714 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx_f3e22a16-2734-45fb-9e1f-cfba0060dfa7/pull/0.log" Jul 12 19:02:22.385532 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:22.385505 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx_f3e22a16-2734-45fb-9e1f-cfba0060dfa7/extract/0.log" Jul 12 19:02:22.494608 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:22.494582 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8_21889432-f78d-4a4a-9061-64d3d0958737/util/0.log" Jul 12 19:02:22.500200 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:22.500176 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8_21889432-f78d-4a4a-9061-64d3d0958737/pull/0.log" Jul 12 19:02:22.505592 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:22.505571 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8_21889432-f78d-4a4a-9061-64d3d0958737/extract/0.log" Jul 12 19:02:22.611071 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:22.611040 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92_fd891299-c307-491b-b9bf-bd6058d101cc/extract/0.log" Jul 12 19:02:22.616442 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:22.616416 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92_fd891299-c307-491b-b9bf-bd6058d101cc/util/0.log" Jul 12 19:02:22.621986 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:22.621962 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92_fd891299-c307-491b-b9bf-bd6058d101cc/pull/0.log" Jul 12 19:02:22.729443 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:22.729363 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f_0c77cdb1-4b62-47b7-b887-363942e724ff/pull/0.log" Jul 12 19:02:22.735209 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:22.735183 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f_0c77cdb1-4b62-47b7-b887-363942e724ff/extract/0.log" Jul 12 19:02:22.740867 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:22.740841 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f_0c77cdb1-4b62-47b7-b887-363942e724ff/util/0.log" Jul 12 19:02:23.073093 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:23.073063 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-kp6lf_9f2d972d-3329-4b94-b5e8-ede7b89f0649/manager/0.log" Jul 12 19:02:23.304816 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:23.304781 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba03b06e-c152-4e85-b78a-66473fb087ea" path="/var/lib/kubelet/pods/ba03b06e-c152-4e85-b78a-66473fb087ea/volumes" Jul 12 19:02:23.529173 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:23.529147 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-45x44_d9455fd6-e2d9-4886-9be8-c554096d4e33/limitador/0.log" Jul 12 19:02:23.645454 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:23.645418 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-kbvzd_bfb07e88-9932-4ef7-9488-57b79c51578f/manager/0.log" Jul 12 19:02:23.982218 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:23.982138 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mdl9w_e80edd8c-d4a2-4bda-9ce6-029183c3ede1/istio-proxy/0.log" Jul 12 19:02:24.097222 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:24.097192 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ncpzc_f3b2bba9-377f-4663-9c77-95f1a1e8910c/discovery/0.log" Jul 12 19:02:24.202761 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:24.202730 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd95989b6-86z8m_f146b5fb-f81f-4a5d-a9b6-82b800bea9ca/kube-auth-proxy/0.log" Jul 12 19:02:24.426058 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:24.426024 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-24lns_fc204fa9-67fb-476a-a6f0-c31680438323/istio-proxy/1.log" Jul 12 19:02:24.775627 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:24.775595 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-549c4cff84-xqpgj_80edfd93-3c7a-4356-9c21-6b1376df14fb/router/0.log" Jul 12 19:02:25.225664 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:25.225576 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-jwjb8_6d60474b-9767-4eb2-892d-5906ff231fea/main/0.log" Jul 12 19:02:29.445442 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.445403 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pxs8x/must-gather-dbzbv"] Jul 12 19:02:29.445901 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.445831 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47968888-9aac-4986-9983-caf8204877fb" containerName="istio-proxy" Jul 12 19:02:29.445901 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.445845 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="47968888-9aac-4986-9983-caf8204877fb" containerName="istio-proxy" Jul 12 19:02:29.445901 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.445863 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerName="istio-proxy" Jul 12 19:02:29.445901 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.445871 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerName="istio-proxy" Jul 12 19:02:29.445901 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.445880 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47968888-9aac-4986-9983-caf8204877fb" containerName="istio-proxy" Jul 12 19:02:29.445901 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.445886 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="47968888-9aac-4986-9983-caf8204877fb" containerName="istio-proxy" Jul 12 19:02:29.445901 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.445894 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerName="istio-proxy" Jul 12 19:02:29.445901 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.445900 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerName="istio-proxy" Jul 12 19:02:29.446409 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.446014 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="47968888-9aac-4986-9983-caf8204877fb" containerName="istio-proxy" Jul 12 19:02:29.446409 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.446032 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerName="istio-proxy" Jul 12 19:02:29.446409 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.446040 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba03b06e-c152-4e85-b78a-66473fb087ea" containerName="istio-proxy" Jul 12 19:02:29.446409 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.446209 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="47968888-9aac-4986-9983-caf8204877fb" containerName="istio-proxy" Jul 12 19:02:29.450801 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.450775 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pxs8x/must-gather-dbzbv" Jul 12 19:02:29.453390 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.453360 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pxs8x\"/\"openshift-service-ca.crt\"" Jul 12 19:02:29.453548 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.453411 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pxs8x\"/\"kube-root-ca.crt\"" Jul 12 19:02:29.454244 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.454225 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pxs8x\"/\"default-dockercfg-6wkmn\"" Jul 12 19:02:29.458470 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.458440 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pxs8x/must-gather-dbzbv"] Jul 12 19:02:29.552491 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.552436 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a07b031a-5ff4-4a90-bdae-f6cadd75d031-must-gather-output\") pod \"must-gather-dbzbv\" (UID: \"a07b031a-5ff4-4a90-bdae-f6cadd75d031\") " pod="openshift-must-gather-pxs8x/must-gather-dbzbv" Jul 12 19:02:29.552684 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.552595 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjbt9\" (UniqueName: \"kubernetes.io/projected/a07b031a-5ff4-4a90-bdae-f6cadd75d031-kube-api-access-bjbt9\") pod \"must-gather-dbzbv\" (UID: \"a07b031a-5ff4-4a90-bdae-f6cadd75d031\") " pod="openshift-must-gather-pxs8x/must-gather-dbzbv" Jul 12 19:02:29.653582 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.653539 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjbt9\" (UniqueName: \"kubernetes.io/projected/a07b031a-5ff4-4a90-bdae-f6cadd75d031-kube-api-access-bjbt9\") pod \"must-gather-dbzbv\" (UID: \"a07b031a-5ff4-4a90-bdae-f6cadd75d031\") " pod="openshift-must-gather-pxs8x/must-gather-dbzbv" Jul 12 19:02:29.653756 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.653615 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a07b031a-5ff4-4a90-bdae-f6cadd75d031-must-gather-output\") pod \"must-gather-dbzbv\" (UID: \"a07b031a-5ff4-4a90-bdae-f6cadd75d031\") " pod="openshift-must-gather-pxs8x/must-gather-dbzbv" Jul 12 19:02:29.653998 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.653980 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a07b031a-5ff4-4a90-bdae-f6cadd75d031-must-gather-output\") pod \"must-gather-dbzbv\" (UID: \"a07b031a-5ff4-4a90-bdae-f6cadd75d031\") " pod="openshift-must-gather-pxs8x/must-gather-dbzbv" Jul 12 19:02:29.662733 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.662697 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjbt9\" (UniqueName: \"kubernetes.io/projected/a07b031a-5ff4-4a90-bdae-f6cadd75d031-kube-api-access-bjbt9\") pod \"must-gather-dbzbv\" (UID: \"a07b031a-5ff4-4a90-bdae-f6cadd75d031\") " pod="openshift-must-gather-pxs8x/must-gather-dbzbv" Jul 12 19:02:29.762769 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.762666 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pxs8x/must-gather-dbzbv" Jul 12 19:02:29.915300 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.915267 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pxs8x/must-gather-dbzbv"] Jul 12 19:02:29.917082 ip-10-0-140-170 kubenswrapper[2585]: W0712 19:02:29.917056 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda07b031a_5ff4_4a90_bdae_f6cadd75d031.slice/crio-a0341afe12090d8a585ecbf48d845d67b32c2862e5a64def17960236d2be8f2b WatchSource:0}: Error finding container a0341afe12090d8a585ecbf48d845d67b32c2862e5a64def17960236d2be8f2b: Status 404 returned error can't find the container with id a0341afe12090d8a585ecbf48d845d67b32c2862e5a64def17960236d2be8f2b Jul 12 19:02:29.919000 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:29.918983 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 19:02:30.652508 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:30.652464 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pxs8x/must-gather-dbzbv" event={"ID":"a07b031a-5ff4-4a90-bdae-f6cadd75d031","Type":"ContainerStarted","Data":"a0341afe12090d8a585ecbf48d845d67b32c2862e5a64def17960236d2be8f2b"} Jul 12 19:02:31.662699 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:31.662591 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pxs8x/must-gather-dbzbv" event={"ID":"a07b031a-5ff4-4a90-bdae-f6cadd75d031","Type":"ContainerStarted","Data":"ae2ba98c4b54afd5cf1360a34d01a0af1187143828c08c1f15f9e8ccedd09dca"} Jul 12 19:02:31.662699 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:31.662655 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pxs8x/must-gather-dbzbv" event={"ID":"a07b031a-5ff4-4a90-bdae-f6cadd75d031","Type":"ContainerStarted","Data":"a020f39245daf7bd4338dc48f2ba7956e2f8a3fa313ecc4734076b2ba07c4621"} Jul 12 19:02:31.681077 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:31.681018 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pxs8x/must-gather-dbzbv" podStartSLOduration=1.747241291 podStartE2EDuration="2.680998315s" totalImagesPullingTime="933.757024ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 19:02:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:02:31.678065615 +0000 UTC m=+2242.877532645" watchObservedRunningTime="2026-07-12 19:02:31.680998315 +0000 UTC m=+2242.880465345" Jul 12 19:02:32.447461 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:32.447430 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j5bpc_6ad0a79f-69ee-49e8-90eb-c27482619b69/global-pull-secret-syncer/0.log" Jul 12 19:02:32.557440 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:32.557404 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cwkh5_5a8d8139-c043-4f8c-b49e-76a54608b5af/konnectivity-agent/0.log" Jul 12 19:02:32.661112 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:32.661079 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-170.ec2.internal_ec39bd46f4e845dad75a1dba09eb9d6f/haproxy/0.log" Jul 12 19:02:36.999485 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:36.999452 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx_f3e22a16-2734-45fb-9e1f-cfba0060dfa7/extract/0.log" Jul 12 19:02:37.022246 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:37.022212 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx_f3e22a16-2734-45fb-9e1f-cfba0060dfa7/util/0.log" Jul 12 19:02:37.043867 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:37.043832 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rtvrx_f3e22a16-2734-45fb-9e1f-cfba0060dfa7/pull/0.log" Jul 12 19:02:37.084542 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:37.084512 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8_21889432-f78d-4a4a-9061-64d3d0958737/extract/0.log" Jul 12 19:02:37.113859 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:37.113784 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8_21889432-f78d-4a4a-9061-64d3d0958737/util/0.log" Jul 12 19:02:37.142882 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:37.142849 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x64z8_21889432-f78d-4a4a-9061-64d3d0958737/pull/0.log" Jul 12 19:02:37.172258 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:37.172230 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92_fd891299-c307-491b-b9bf-bd6058d101cc/extract/0.log" Jul 12 19:02:37.196134 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:37.196105 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92_fd891299-c307-491b-b9bf-bd6058d101cc/util/0.log" Jul 12 19:02:37.221823 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:37.221792 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmc92_fd891299-c307-491b-b9bf-bd6058d101cc/pull/0.log" Jul 12 19:02:37.256079 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:37.255965 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f_0c77cdb1-4b62-47b7-b887-363942e724ff/extract/0.log" Jul 12 19:02:37.279410 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:37.279375 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f_0c77cdb1-4b62-47b7-b887-363942e724ff/util/0.log" Jul 12 19:02:37.312020 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:37.311966 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1zxr7f_0c77cdb1-4b62-47b7-b887-363942e724ff/pull/0.log" Jul 12 19:02:37.630830 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:37.630803 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-kp6lf_9f2d972d-3329-4b94-b5e8-ede7b89f0649/manager/0.log" Jul 12 19:02:37.898490 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:37.898396 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-45x44_d9455fd6-e2d9-4886-9be8-c554096d4e33/limitador/0.log" Jul 12 19:02:37.970554 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:37.970508 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-kbvzd_bfb07e88-9932-4ef7-9488-57b79c51578f/manager/0.log" Jul 12 19:02:39.778359 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:39.778326 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-642k2_a33fa508-623e-4693-88b0-51eb8cc79715/cluster-monitoring-operator/0.log" Jul 12 19:02:39.947774 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:39.947740 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b6f7f_8d3f6345-96f3-40ba-9a77-73c5e424e968/node-exporter/0.log" Jul 12 19:02:39.971105 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:39.971032 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b6f7f_8d3f6345-96f3-40ba-9a77-73c5e424e968/kube-rbac-proxy/0.log" Jul 12 19:02:39.993879 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:39.993843 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b6f7f_8d3f6345-96f3-40ba-9a77-73c5e424e968/init-textfile/0.log" Jul 12 19:02:42.172538 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:42.172495 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-kzl7d_d1e3d19c-3412-49a9-a51f-4658c02209bd/networking-console-plugin/0.log" Jul 12 19:02:42.808906 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:42.808879 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/1.log" Jul 12 19:02:42.815780 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:42.815743 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9nvfk_23bc3496-ee12-455a-8e8a-6101006dbcb7/console-operator/2.log" Jul 12 19:02:43.262781 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.262678 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w"] Jul 12 19:02:43.271584 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.271547 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.277090 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.277057 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w"] Jul 12 19:02:43.397161 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.397124 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5fabde5c-0b33-4a97-8ce5-49991788da96-proc\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.397161 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.397177 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5fabde5c-0b33-4a97-8ce5-49991788da96-lib-modules\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.397442 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.397205 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fabde5c-0b33-4a97-8ce5-49991788da96-sys\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.397442 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.397238 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5fabde5c-0b33-4a97-8ce5-49991788da96-podres\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.397442 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.397266 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr8ns\" (UniqueName: \"kubernetes.io/projected/5fabde5c-0b33-4a97-8ce5-49991788da96-kube-api-access-mr8ns\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.423319 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.423281 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-gzh72_484975e5-3874-48ae-a247-8c9b36bc813d/download-server/0.log" Jul 12 19:02:43.499199 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.498154 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5fabde5c-0b33-4a97-8ce5-49991788da96-proc\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.499199 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.498243 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5fabde5c-0b33-4a97-8ce5-49991788da96-lib-modules\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.499199 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.498277 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fabde5c-0b33-4a97-8ce5-49991788da96-sys\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.499199 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.498304 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5fabde5c-0b33-4a97-8ce5-49991788da96-podres\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.499199 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.498334 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr8ns\" (UniqueName: \"kubernetes.io/projected/5fabde5c-0b33-4a97-8ce5-49991788da96-kube-api-access-mr8ns\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.499199 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.498847 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5fabde5c-0b33-4a97-8ce5-49991788da96-proc\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.499199 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.499015 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5fabde5c-0b33-4a97-8ce5-49991788da96-lib-modules\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.499199 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.499069 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5fabde5c-0b33-4a97-8ce5-49991788da96-sys\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.499199 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.499156 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5fabde5c-0b33-4a97-8ce5-49991788da96-podres\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.507060 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.506985 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr8ns\" (UniqueName: \"kubernetes.io/projected/5fabde5c-0b33-4a97-8ce5-49991788da96-kube-api-access-mr8ns\") pod \"perf-node-gather-daemonset-dqq4w\" (UID: \"5fabde5c-0b33-4a97-8ce5-49991788da96\") " pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.593300 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.593254 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:43.778806 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:43.776575 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w"] Jul 12 19:02:43.784304 ip-10-0-140-170 kubenswrapper[2585]: W0712 19:02:43.784232 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5fabde5c_0b33_4a97_8ce5_49991788da96.slice/crio-c96a6cfd05d04f1f8c9fe6101b0ffe73baf9f6504242073aa96097a13a71ba7b WatchSource:0}: Error finding container c96a6cfd05d04f1f8c9fe6101b0ffe73baf9f6504242073aa96097a13a71ba7b: Status 404 returned error can't find the container with id c96a6cfd05d04f1f8c9fe6101b0ffe73baf9f6504242073aa96097a13a71ba7b Jul 12 19:02:44.053224 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:44.053191 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-9cft2_ca704d74-c4dc-4547-b226-2f910224db6c/volume-data-source-validator/0.log" Jul 12 19:02:44.794642 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:44.794608 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" event={"ID":"5fabde5c-0b33-4a97-8ce5-49991788da96","Type":"ContainerStarted","Data":"031b1c8a5c4172e7cd6349f30bf7fe00f8362151b30b4924792f85398f7411cc"} Jul 12 19:02:44.795090 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:44.794668 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" event={"ID":"5fabde5c-0b33-4a97-8ce5-49991788da96","Type":"ContainerStarted","Data":"c96a6cfd05d04f1f8c9fe6101b0ffe73baf9f6504242073aa96097a13a71ba7b"} Jul 12 19:02:44.795090 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:44.794759 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:45.115657 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:45.115576 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lphwv_cb62dd7e-0d0f-4128-87dd-3158aa45a396/dns/0.log" Jul 12 19:02:45.136209 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:45.136181 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lphwv_cb62dd7e-0d0f-4128-87dd-3158aa45a396/kube-rbac-proxy/0.log" Jul 12 19:02:45.228241 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:45.228215 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x7b84_fa50005b-ad03-474a-a648-f9fde6a932a6/dns-node-resolver/0.log" Jul 12 19:02:45.823781 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:45.823726 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5dzg6_64c451f1-c731-40c5-a837-b515e65009e0/node-ca/0.log" Jul 12 19:02:46.756108 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:46.756067 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mdl9w_e80edd8c-d4a2-4bda-9ce6-029183c3ede1/istio-proxy/0.log" Jul 12 19:02:46.882159 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:46.882133 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ncpzc_f3b2bba9-377f-4663-9c77-95f1a1e8910c/discovery/0.log" Jul 12 19:02:46.901351 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:46.901324 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd95989b6-86z8m_f146b5fb-f81f-4a5d-a9b6-82b800bea9ca/kube-auth-proxy/0.log" Jul 12 19:02:46.953973 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:46.953940 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-24lns_fc204fa9-67fb-476a-a6f0-c31680438323/istio-proxy/0.log" Jul 12 19:02:46.992541 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:46.992507 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-24lns_fc204fa9-67fb-476a-a6f0-c31680438323/istio-proxy/1.log" Jul 12 19:02:47.149617 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:47.149579 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-549c4cff84-xqpgj_80edfd93-3c7a-4356-9c21-6b1376df14fb/router/0.log" Jul 12 19:02:47.868729 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:47.868692 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l98hb_9aef0f72-416e-4c37-98b8-a68e30a65ce6/serve-healthcheck-canary/0.log" Jul 12 19:02:48.355984 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:48.355925 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-96sxt_b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2/insights-operator/0.log" Jul 12 19:02:48.356533 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:48.356521 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-96sxt_b6e11c33-f3a3-4d9d-8d5b-1cd95defcea2/insights-operator/1.log" Jul 12 19:02:48.501098 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:48.501068 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r5d4d_e98e68e3-3c5c-45b0-9817-a05d41b96502/kube-rbac-proxy/0.log" Jul 12 19:02:48.541451 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:48.541409 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r5d4d_e98e68e3-3c5c-45b0-9817-a05d41b96502/exporter/0.log" Jul 12 19:02:48.583466 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:48.583431 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r5d4d_e98e68e3-3c5c-45b0-9817-a05d41b96502/extractor/0.log" Jul 12 19:02:50.816806 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:50.816774 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" Jul 12 19:02:50.833307 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:50.833246 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pxs8x/perf-node-gather-daemonset-dqq4w" podStartSLOduration=7.833230245 podStartE2EDuration="7.833230245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 19:02:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:02:44.80945849 +0000 UTC m=+2256.008925520" watchObservedRunningTime="2026-07-12 19:02:50.833230245 +0000 UTC m=+2262.032697272" Jul 12 19:02:51.128610 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:51.128524 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-l5nxv_b96066fe-ee3b-40a9-80fb-e1fd4bbeac68/server/0.log" Jul 12 19:02:51.199831 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:51.199789 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6f7h9_59e7df2c-c8b5-4163-9bc7-baa9125014df/manager/0.log" Jul 12 19:02:51.268819 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:51.268784 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f88dd4c8-l6j59_3d6039e0-174a-4524-ba46-500c1604c808/manager/0.log" Jul 12 19:02:51.288631 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:51.288605 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59f88dd4c8-l6j59_3d6039e0-174a-4524-ba46-500c1604c808/copy-manifests/0.log" Jul 12 19:02:52.761805 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:52.761754 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-66j6r_1c0620f9-b6aa-4571-859e-497759d11f5c/openshift-lws-operator/0.log" Jul 12 19:02:54.274685 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:54.274650 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mdl9w_e80edd8c-d4a2-4bda-9ce6-029183c3ede1/istio-proxy/0.log" Jul 12 19:02:54.396840 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:54.396801 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ncpzc_f3b2bba9-377f-4663-9c77-95f1a1e8910c/discovery/0.log" Jul 12 19:02:54.408569 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:54.408534 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd95989b6-86z8m_f146b5fb-f81f-4a5d-a9b6-82b800bea9ca/kube-auth-proxy/0.log" Jul 12 19:02:54.446646 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:54.446610 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-24lns_fc204fa9-67fb-476a-a6f0-c31680438323/istio-proxy/0.log" Jul 12 19:02:54.485459 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:54.485416 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-24lns_fc204fa9-67fb-476a-a6f0-c31680438323/istio-proxy/1.log" Jul 12 19:02:54.614253 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:54.614172 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-549c4cff84-xqpgj_80edfd93-3c7a-4356-9c21-6b1376df14fb/router/0.log" Jul 12 19:02:55.942792 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:55.942706 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mdl9w_e80edd8c-d4a2-4bda-9ce6-029183c3ede1/istio-proxy/0.log" Jul 12 19:02:56.055429 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:56.055398 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ncpzc_f3b2bba9-377f-4663-9c77-95f1a1e8910c/discovery/0.log" Jul 12 19:02:56.068703 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:56.068667 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dd95989b6-86z8m_f146b5fb-f81f-4a5d-a9b6-82b800bea9ca/kube-auth-proxy/0.log" Jul 12 19:02:56.109314 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:56.109285 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-24lns_fc204fa9-67fb-476a-a6f0-c31680438323/istio-proxy/0.log" Jul 12 19:02:56.149712 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:56.149660 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-24lns_fc204fa9-67fb-476a-a6f0-c31680438323/istio-proxy/1.log" Jul 12 19:02:56.286187 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:56.286153 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-549c4cff84-xqpgj_80edfd93-3c7a-4356-9c21-6b1376df14fb/router/0.log" Jul 12 19:02:58.608283 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:58.608220 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-pm7l6_429df8ad-28e9-487c-a2d9-d66a44615bc4/kube-storage-version-migrator-operator/1.log" Jul 12 19:02:58.609063 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:58.609044 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-pm7l6_429df8ad-28e9-487c-a2d9-d66a44615bc4/kube-storage-version-migrator-operator/0.log" Jul 12 19:02:59.689206 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:59.689178 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9ph4f_c4122547-ea68-4da7-af41-996662132ed6/kube-multus/0.log" Jul 12 19:02:59.862766 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:59.862734 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-llfzt_dd93dddb-eaaa-470d-84a4-17c4f61493e8/kube-multus-additional-cni-plugins/0.log" Jul 12 19:02:59.882263 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:59.882225 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-llfzt_dd93dddb-eaaa-470d-84a4-17c4f61493e8/egress-router-binary-copy/0.log" Jul 12 19:02:59.901395 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:59.901365 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-llfzt_dd93dddb-eaaa-470d-84a4-17c4f61493e8/cni-plugins/0.log" Jul 12 19:02:59.921622 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:59.921593 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-llfzt_dd93dddb-eaaa-470d-84a4-17c4f61493e8/bond-cni-plugin/0.log" Jul 12 19:02:59.941384 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:59.941305 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-llfzt_dd93dddb-eaaa-470d-84a4-17c4f61493e8/routeoverride-cni/0.log" Jul 12 19:02:59.961675 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:59.961642 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-llfzt_dd93dddb-eaaa-470d-84a4-17c4f61493e8/whereabouts-cni-bincopy/0.log" Jul 12 19:02:59.981209 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:02:59.981179 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-llfzt_dd93dddb-eaaa-470d-84a4-17c4f61493e8/whereabouts-cni/0.log" Jul 12 19:03:00.344651 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:03:00.344613 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rgkpj_87644bbc-d561-4ca5-8b2b-38bf70f01efb/network-metrics-daemon/0.log" Jul 12 19:03:00.363625 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:03:00.363599 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rgkpj_87644bbc-d561-4ca5-8b2b-38bf70f01efb/kube-rbac-proxy/0.log" Jul 12 19:03:01.209302 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:03:01.209197 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gswlt_e0391cb3-5bc7-4fb5-9d22-f831b4125304/ovn-controller/0.log" Jul 12 19:03:01.225277 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:03:01.225249 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gswlt_e0391cb3-5bc7-4fb5-9d22-f831b4125304/ovn-acl-logging/0.log" Jul 12 19:03:01.249789 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:03:01.249750 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gswlt_e0391cb3-5bc7-4fb5-9d22-f831b4125304/kube-rbac-proxy-node/0.log" Jul 12 19:03:01.269873 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:03:01.269849 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gswlt_e0391cb3-5bc7-4fb5-9d22-f831b4125304/kube-rbac-proxy-ovn-metrics/0.log" Jul 12 19:03:01.288643 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:03:01.288615 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gswlt_e0391cb3-5bc7-4fb5-9d22-f831b4125304/northd/0.log" Jul 12 19:03:01.313064 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:03:01.313031 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gswlt_e0391cb3-5bc7-4fb5-9d22-f831b4125304/nbdb/0.log" Jul 12 19:03:01.334168 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:03:01.334138 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gswlt_e0391cb3-5bc7-4fb5-9d22-f831b4125304/sbdb/0.log" Jul 12 19:03:01.395982 ip-10-0-140-170 kubenswrapper[2585]: I0712 19:03:01.395910 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gswlt_e0391cb3-5bc7-4fb5-9d22-f831b4125304/ovnkube-controller/0.log"