Jul 06 20:05:46.761145 ip-10-0-130-133 systemd[1]: Starting Kubernetes Kubelet... Jul 06 20:05:47.157617 ip-10-0-130-133 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 20:05:47.157617 ip-10-0-130-133 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 20:05:47.157617 ip-10-0-130-133 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 20:05:47.157617 ip-10-0-130-133 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 20:05:47.157617 ip-10-0-130-133 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 20:05:47.157617 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.132016 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 20:05:47.157617 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136143 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:05:47.157617 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136154 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:05:47.157617 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136158 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:05:47.157617 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136161 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:05:47.157617 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136164 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:05:47.157617 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136167 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:05:47.157617 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136169 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:05:47.157617 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136172 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136175 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136177 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136180 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136182 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136185 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136187 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136191 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136194 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136196 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136199 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136201 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136204 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136207 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136210 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136212 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136224 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136228 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136231 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136236 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:05:47.158330 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136239 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136242 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136245 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136249 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136252 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136255 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136257 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136260 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136262 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136265 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136268 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136271 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136273 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136276 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136279 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136283 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136286 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136289 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136292 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:05:47.158917 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136295 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136298 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136300 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136303 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136305 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136308 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136310 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136313 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136316 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136319 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136322 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136325 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136328 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136330 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136333 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136336 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136339 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136342 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136344 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136347 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:05:47.159549 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136349 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136352 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136355 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136357 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136359 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136363 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136365 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136368 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136370 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136373 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136375 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136378 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136380 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136383 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136385 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136388 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136391 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136393 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136395 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136398 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:05:47.160136 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136400 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136403 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136406 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136409 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136411 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136415 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136418 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136422 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136424 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136427 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136429 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136432 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.136434 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137121 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137127 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137130 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137135 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137139 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137143 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:05:47.160658 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137147 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137150 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137153 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137156 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137159 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137162 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137164 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137167 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137169 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137172 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137174 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137178 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137180 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137183 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137187 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137191 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137194 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137197 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137200 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137202 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:05:47.161291 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137205 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137207 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137210 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137213 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137215 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137218 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137220 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137222 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137225 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137227 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137230 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137233 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137236 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137239 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137241 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137244 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137248 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137250 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137253 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137255 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:05:47.161838 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137257 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137260 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137262 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137265 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137267 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137270 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137273 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137275 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137278 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137280 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137283 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137285 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137289 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137292 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137294 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137297 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137299 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137301 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137304 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137306 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:05:47.162432 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137308 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137311 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137313 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137316 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137318 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137321 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137324 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137328 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137332 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137335 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137337 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137340 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137343 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137346 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137348 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137351 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137355 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137358 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137360 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:05:47.163289 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137363 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137366 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137368 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137371 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137373 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137375 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137378 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137380 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137383 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137385 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137387 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137390 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137392 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137395 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137744 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137748 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137751 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137753 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137757 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137759 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:05:47.164174 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137762 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137765 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137767 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137770 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137772 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137775 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137778 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137780 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137783 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137788 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137791 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137793 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137796 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137798 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137801 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137804 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137806 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137810 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137814 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:05:47.165097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137817 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137820 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137822 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137825 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137827 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137830 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137833 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137835 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137838 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137842 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137844 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137847 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137850 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137852 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137854 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137857 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137859 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137862 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137864 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137868 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:05:47.166142 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137887 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137890 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137895 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137898 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137900 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137903 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137906 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137909 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137911 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137914 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137917 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137919 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137922 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137924 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137927 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137930 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137932 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137935 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137937 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:05:47.167017 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137940 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137942 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137945 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137947 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137949 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137953 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137956 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137958 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137961 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137963 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137966 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137968 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137970 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137974 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137977 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137983 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137985 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137988 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137991 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:05:47.167910 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137993 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137996 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.137998 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138001 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138003 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138006 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138008 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138010 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138013 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138015 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138018 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138020 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138023 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138026 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138028 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138030 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138097 2583 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138103 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138110 2583 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138114 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138119 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138122 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 20:05:47.168770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138126 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138130 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138133 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138136 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138139 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138142 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138148 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138150 2583 flags.go:64] FLAG: --cgroup-root="" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138153 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138156 2583 flags.go:64] FLAG: --client-ca-file="" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138159 2583 flags.go:64] FLAG: --cloud-provider="external" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138162 2583 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138165 2583 flags.go:64] FLAG: --cluster-domain="" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138168 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138171 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138175 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138178 2583 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138181 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138184 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138187 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138191 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138194 2583 flags.go:64] FLAG: --contention-profiling="false" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138196 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138199 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 20:05:47.169836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138202 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138205 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138209 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138212 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138215 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138217 2583 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138220 2583 flags.go:64] FLAG: --enable-server="true" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138224 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138228 2583 flags.go:64] FLAG: --event-burst="100" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138231 2583 flags.go:64] FLAG: --event-qps="50" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138234 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138237 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138240 2583 flags.go:64] FLAG: --eviction-hard="" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138243 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138248 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138251 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138254 2583 flags.go:64] FLAG: --eviction-soft="" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138256 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138259 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138262 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138265 2583 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138268 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138271 2583 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138273 2583 flags.go:64] FLAG: --feature-gates="" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138277 2583 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 20:05:47.171142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138280 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138282 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138285 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138288 2583 flags.go:64] FLAG: --healthz-port="10248" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138291 2583 flags.go:64] FLAG: --help="false" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138294 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138297 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138300 2583 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138302 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138305 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138309 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138312 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138315 2583 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138317 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138320 2583 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138323 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138326 2583 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138329 2583 flags.go:64] FLAG: --kube-reserved="" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138332 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138335 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138338 2583 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138343 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138345 2583 flags.go:64] FLAG: --lock-file="" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138348 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 20:05:47.172220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138351 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138354 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138359 2583 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138362 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138365 2583 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138368 2583 flags.go:64] FLAG: --logging-format="text" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138370 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138373 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138376 2583 flags.go:64] FLAG: --manifest-url="" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138379 2583 flags.go:64] FLAG: --manifest-url-header="" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138383 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138386 2583 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138390 2583 flags.go:64] FLAG: --max-pods="110" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138393 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138396 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138399 2583 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138402 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138405 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138407 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138410 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138416 2583 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138419 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138422 2583 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 20:05:47.176553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138425 2583 flags.go:64] FLAG: --pod-cidr="" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138428 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138433 2583 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138436 2583 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138439 2583 flags.go:64] FLAG: --pods-per-core="0" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138441 2583 flags.go:64] FLAG: --port="10250" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138445 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138449 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0593df0767118269b" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138452 2583 flags.go:64] FLAG: --qos-reserved="" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138455 2583 flags.go:64] FLAG: --read-only-port="10255" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138458 2583 flags.go:64] FLAG: --register-node="true" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138461 2583 flags.go:64] FLAG: --register-with-taints="" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138465 2583 flags.go:64] FLAG: --registry-burst="10" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138468 2583 flags.go:64] FLAG: --registry-qps="5" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138470 2583 flags.go:64] FLAG: --reserved-cpus="" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138473 2583 flags.go:64] FLAG: --reserved-memory="" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138476 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138479 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138482 2583 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138485 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138488 2583 flags.go:64] FLAG: --runonce="false" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138490 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138493 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138496 2583 flags.go:64] FLAG: --seccomp-default="false" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138499 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 20:05:47.177635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138501 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138504 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138507 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138510 2583 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138512 2583 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138515 2583 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138518 2583 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138521 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138524 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138527 2583 flags.go:64] FLAG: --system-cgroups="" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138529 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138546 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138549 2583 flags.go:64] FLAG: --tls-cert-file="" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138553 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138558 2583 flags.go:64] FLAG: --tls-min-version="" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138561 2583 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138563 2583 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138566 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138569 2583 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138572 2583 flags.go:64] FLAG: --v="2" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138576 2583 flags.go:64] FLAG: --version="false" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138579 2583 flags.go:64] FLAG: --vmodule="" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138583 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.138586 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138702 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:05:47.258733 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138706 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:05:47.226328 ip-10-0-130-133 systemd[1]: Started Kubernetes Kubelet. Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138709 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138712 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138715 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138718 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138721 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138724 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138726 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138729 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138732 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138735 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138737 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138740 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138742 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138745 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138748 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138750 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138753 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138756 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138758 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:05:47.259824 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138763 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138766 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138769 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138771 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138774 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138776 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138779 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138781 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138784 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138787 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138789 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138792 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138794 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138797 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138799 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138802 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138805 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138807 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138809 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138812 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:05:47.260897 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138815 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138818 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138820 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138823 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138825 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138828 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138830 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138835 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138838 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138841 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138844 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138847 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138851 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138855 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138857 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138860 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138863 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138865 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138883 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:05:47.261682 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138886 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138889 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138892 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138895 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138898 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138901 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138904 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138907 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138909 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138912 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138914 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138917 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138920 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138922 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138925 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138928 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138930 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138933 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138935 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138938 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:05:47.262471 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138940 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138943 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138946 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138948 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138951 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138955 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138959 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138961 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138965 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138968 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138971 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138974 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138977 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138979 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138982 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138984 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138987 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138990 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:05:47.263312 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.138992 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.139000 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.145457 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.145474 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.146192 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.146207 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146271 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146276 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146279 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146282 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146285 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146288 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146291 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146294 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146297 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:05:47.263963 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146299 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146302 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146304 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146307 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146310 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146313 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146315 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146318 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146321 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146324 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146327 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146329 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146332 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146335 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146337 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146340 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146342 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146345 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146347 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146352 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:05:47.264574 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146361 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146364 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146366 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146369 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146372 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146376 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146380 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146383 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146386 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146389 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146392 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146395 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146398 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146400 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146403 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146406 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146409 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146412 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:05:47.265213 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146416 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146419 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146422 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146424 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146429 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146432 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146436 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146438 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146442 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146444 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146447 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146450 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146453 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146460 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146462 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146472 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146475 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146479 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146481 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146484 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:05:47.265845 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146487 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146489 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146492 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146494 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146497 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146499 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146502 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146505 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146507 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146510 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146512 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146515 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146517 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146520 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146522 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146525 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146527 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146530 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146543 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146547 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:05:47.267287 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146550 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146554 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146557 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146561 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146564 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146570 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146573 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146577 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146588 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146596 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146600 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146604 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.146610 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146827 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146835 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 20:05:47.268344 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146841 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146845 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146849 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146853 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146857 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146861 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146864 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146868 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146889 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146893 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146896 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146900 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146904 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146907 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146911 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146916 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146922 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146926 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146930 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146934 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 20:05:47.268977 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146939 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146943 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146948 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146952 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146956 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146960 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146971 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146977 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146981 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146985 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146988 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146992 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146996 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.146999 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147003 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147007 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147011 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147014 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147018 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 20:05:47.269800 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147022 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147026 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147029 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147033 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147037 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147041 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147045 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147048 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147054 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147058 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147062 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147066 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147070 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147074 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147078 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147083 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147087 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147090 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147093 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147097 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 20:05:47.270637 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147109 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147113 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147116 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147120 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147124 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147128 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147132 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147136 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147139 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147142 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147146 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147150 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147154 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147157 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147160 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147164 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147167 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147171 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147175 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147178 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 20:05:47.271460 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147182 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147185 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147189 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147192 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147196 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147200 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147203 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147208 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147212 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147216 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147220 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147224 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147229 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147239 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147243 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147247 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147250 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 20:05:47.272285 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:47.147255 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 20:05:47.272910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.147262 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 20:05:47.272910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.147408 2583 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 20:05:47.272910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.150007 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 20:05:47.272910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.151481 2583 server.go:1019] "Starting client certificate rotation" Jul 06 20:05:47.272910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.151585 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 20:05:47.272910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.151621 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 20:05:47.272910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.155410 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 20:05:47.272910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.157342 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 20:05:47.272910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.194246 2583 log.go:25] "Validated CRI v1 runtime API" Jul 06 20:05:47.272910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.196389 2583 log.go:25] "Validated CRI v1 image API" Jul 06 20:05:47.272910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.197769 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 20:05:47.272910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.199821 2583 fs.go:135] Filesystem UUIDs: map[1621e1f2-05dd-4ed2-9099-885d21552043:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d766428b-c751-464b-a33a-25ec0a04f6e9:/dev/nvme0n1p3] Jul 06 20:05:47.273340 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.199839 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 20:05:47.273340 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.203709 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.205410 2583 manager.go:216] Machine: {Timestamp:2026-07-06 20:05:47.203528953 +0000 UTC m=+0.330441049 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093400 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2eb11eda1523e44eb469dfac2bcc20 SystemUUID:ec2eb11e-da15-23e4-4eb4-69dfac2bcc20 BootID:16e39dd6-aba0-4a8f-8406-cf225251ba01 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:48:e1:fc:22:35 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:48:e1:fc:22:35 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:53:85:c9:0e:3b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.205511 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.205594 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.208430 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.208452 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-133.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.209101 2583 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.209110 2583 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.209117 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.209675 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.211059 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.215250 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.215361 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.215391 2583 kubelet.go:486] "Attempting to sync node with API server" Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.215407 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.215421 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.215429 2583 kubelet.go:398] "Adding apiserver pod source" Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.215437 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 20:05:47.274836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.215760 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.215775 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.217602 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.219704 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.219735 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.220824 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.220837 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.220843 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.220862 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.220890 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.220897 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.220905 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.220910 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.220916 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.220922 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.220934 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.220939 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.221565 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.221572 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.224626 2583 server.go:1268] "Started kubelet" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.224757 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.225260 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.225329 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.227063 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.229216 2583 server.go:310] "Adding debug handlers to kubelet server" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.231341 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.234517 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.234592 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.234646 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.235185 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.235199 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.236740 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-133.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.235702 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.240917 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.240975 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.240989 2583 reconstruct.go:97] "Volume reconstruction finished" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.240995 2583 reconciler.go:29] "Reconciler: start to sync state" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.236773 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-133.ec2.internal.18bfcc1027ff4ec9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-133.ec2.internal,UID:ip-10-0-130-133.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-133.ec2.internal,},FirstTimestamp:2026-07-06 20:05:47.224592073 +0000 UTC m=+0.351504168,LastTimestamp:2026-07-06 20:05:47.224592073 +0000 UTC m=+0.351504168,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-133.ec2.internal,}" Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.242448 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.242458 2583 factory.go:55] Registering systemd factory Jul 06 20:05:47.275646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.242464 2583 factory.go:223] Registration of the systemd container factory successfully Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.242686 2583 factory.go:153] Registering CRI-O factory Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.242699 2583 factory.go:223] Registration of the crio container factory successfully Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.242715 2583 factory.go:103] Registering Raw factory Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.242728 2583 manager.go:1196] Started watching for new ooms in manager Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.243038 2583 manager.go:349] Starting recovery of all containers Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.248854 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.248987 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.252692 2583 manager.go:354] Recovery completed Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.260414 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wdqbq" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.263806 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.263919 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wdqbq" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.265803 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.265835 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.265850 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.267828 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.267842 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.267860 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.267867 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.268855 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.268906 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.268921 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.273361 2583 policy_none.go:49] "None policy: Start" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.273389 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.273403 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.273434 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-133.ec2.internal.18bfcc102a7458c3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-133.ec2.internal,UID:ip-10-0-130-133.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-133.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-133.ec2.internal,},FirstTimestamp:2026-07-06 20:05:47.265816771 +0000 UTC m=+0.392728866,LastTimestamp:2026-07-06 20:05:47.265816771 +0000 UTC m=+0.392728866,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-133.ec2.internal,}" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.275625 2583 policy_none.go:47] "Start" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.329749 2583 manager.go:337] "Starting Device Plugin manager" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.329890 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.329904 2583 server.go:87] "Starting device plugin registration server" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.330109 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.330118 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.330223 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.330293 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.330302 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.330688 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 20:05:47.386511 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.330737 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-133.ec2.internal\" not found" Jul 06 20:05:47.541126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.431045 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:05:47.541126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.431781 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:05:47.541126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.431807 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:05:47.541126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.431818 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:05:47.541126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.431841 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.541126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.447227 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.541126 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.447243 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-133.ec2.internal\": node \"ip-10-0-130-133.ec2.internal\" not found" Jul 06 20:05:47.541126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.477087 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 20:05:47.541126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.478306 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 20:05:47.541126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.478337 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 20:05:47.541126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.478358 2583 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 20:05:47.541126 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.478392 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 20:05:47.541126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.482467 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:05:47.541126 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.489708 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.580309 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal"] Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.580369 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.581187 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.581208 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.581218 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.583813 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.583967 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.583996 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.584477 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.584473 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.584505 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.584508 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.584516 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.584519 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.587067 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.587087 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.587724 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientMemory" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.587748 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.587758 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientPID" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.590331 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.619497 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-133.ec2.internal\" not found" node="ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.624570 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-133.ec2.internal\" not found" node="ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.646329 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d927aedbafdf82c88fcd9938008cb116-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal\" (UID: \"d927aedbafdf82c88fcd9938008cb116\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.646353 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d927aedbafdf82c88fcd9938008cb116-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal\" (UID: \"d927aedbafdf82c88fcd9938008cb116\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.661739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.646370 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/19411eae83bc603ba88b7f2105ff736e-config\") pod \"kube-apiserver-proxy-ip-10-0-130-133.ec2.internal\" (UID: \"19411eae83bc603ba88b7f2105ff736e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.793417 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.692143 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found" Jul 06 20:05:47.793417 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.746558 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d927aedbafdf82c88fcd9938008cb116-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal\" (UID: \"d927aedbafdf82c88fcd9938008cb116\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.793417 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.746609 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d927aedbafdf82c88fcd9938008cb116-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal\" (UID: \"d927aedbafdf82c88fcd9938008cb116\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.793417 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.746652 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d927aedbafdf82c88fcd9938008cb116-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal\" (UID: \"d927aedbafdf82c88fcd9938008cb116\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.793417 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.746675 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/19411eae83bc603ba88b7f2105ff736e-config\") pod \"kube-apiserver-proxy-ip-10-0-130-133.ec2.internal\" (UID: \"19411eae83bc603ba88b7f2105ff736e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.793417 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.746710 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/19411eae83bc603ba88b7f2105ff736e-config\") pod \"kube-apiserver-proxy-ip-10-0-130-133.ec2.internal\" (UID: \"19411eae83bc603ba88b7f2105ff736e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.793417 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.746749 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d927aedbafdf82c88fcd9938008cb116-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal\" (UID: \"d927aedbafdf82c88fcd9938008cb116\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.793417 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.793234 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found" Jul 06 20:05:47.927489 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.894010 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found" Jul 06 20:05:47.927489 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.924430 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" Jul 06 20:05:47.928641 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:47.928218 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal" Jul 06 20:05:48.071710 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:47.997105 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found" Jul 06 20:05:48.071710 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:48.067616 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:05:48.213433 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:48.097953 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found" Jul 06 20:05:48.213433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:48.151277 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 20:05:48.213433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:48.151390 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 20:05:48.213433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:48.151423 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 20:05:48.213433 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:48.200571 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found" Jul 06 20:05:48.361492 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:48.268231 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 20:00:47 +0000 UTC" deadline="2028-01-13 06:40:27.743875976 +0000 UTC" Jul 06 20:05:48.361492 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:48.268252 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13330h34m39.475626934s" Jul 06 20:05:48.361492 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:48.301634 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found" Jul 06 20:05:48.518003 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:48.401918 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found" Jul 06 20:05:48.518003 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:48.472306 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:05:48.670257 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:48.542318 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" Jul 06 20:05:48.670257 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:48.577975 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 20:05:48.670257 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:48.578923 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal" Jul 06 20:05:48.670257 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:48.599148 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 20:05:48.670257 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:48.612662 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:05:49.159194 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:49.079617 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd927aedbafdf82c88fcd9938008cb116.slice/crio-b80cf11967ac48364cffbb3b2f7034efa5ea1e647d81a5ca7da168beab73d60c WatchSource:0}: Error finding container b80cf11967ac48364cffbb3b2f7034efa5ea1e647d81a5ca7da168beab73d60c: Status 404 returned error can't find the container with id b80cf11967ac48364cffbb3b2f7034efa5ea1e647d81a5ca7da168beab73d60c Jul 06 20:05:49.159194 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.102035 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:05:49.159194 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:49.134146 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19411eae83bc603ba88b7f2105ff736e.slice/crio-3f36f91999d06c0fa426f624e7bc050a057b5900b7d34a110ad15ce36ac94586 WatchSource:0}: Error finding container 3f36f91999d06c0fa426f624e7bc050a057b5900b7d34a110ad15ce36ac94586: Status 404 returned error can't find the container with id 3f36f91999d06c0fa426f624e7bc050a057b5900b7d34a110ad15ce36ac94586 Jul 06 20:05:49.220447 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.220426 2583 apiserver.go:52] "Watching apiserver" Jul 06 20:05:49.230176 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.230155 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 20:05:49.232488 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.232467 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-7t42j","openshift-multus/multus-additional-cni-plugins-8jp5d","openshift-multus/network-metrics-daemon-ztl9r","kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal","openshift-image-registry/node-ca-wm5jd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal","openshift-network-diagnostics/network-check-target-spktr","openshift-network-operator/iptables-alerter-nk5s5","openshift-ovn-kubernetes/ovnkube-node-4m62r","kube-system/global-pull-secret-syncer-r6ptl","kube-system/konnectivity-agent-wvqlx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4","openshift-cluster-node-tuning-operator/tuned-65r94","openshift-dns/node-resolver-mpdph"] Jul 06 20:05:49.235221 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.235205 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 20:05:49.235350 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.235330 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nk5s5" Jul 06 20:05:49.237584 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.237566 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.238603 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.238576 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 20:05:49.238792 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.238774 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:05:49.239472 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.239453 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 20:05:49.239738 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.239723 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d9v86\"" Jul 06 20:05:49.239920 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.239903 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:49.240022 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.239966 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:05:49.242300 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.242281 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.244614 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.244596 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.246177 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.246159 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 20:05:49.246246 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.246198 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 20:05:49.246312 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.246272 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 20:05:49.246370 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.246335 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 20:05:49.246473 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.246458 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 20:05:49.246918 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.246903 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wvqlx" Jul 06 20:05:49.249376 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.249257 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:05:49.249376 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.249318 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:05:49.249826 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.249809 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 20:05:49.250456 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.250437 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:05:49.250456 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.250450 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 20:05:49.250648 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.250567 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xr74p\"" Jul 06 20:05:49.250821 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.250806 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-g847p\"" Jul 06 20:05:49.254041 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.253800 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7pxfd\"" Jul 06 20:05:49.254041 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.253866 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.254372 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.254352 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs\") pod \"network-metrics-daemon-ztl9r\" (UID: \"8aa84d62-1d3e-4c96-8f8b-d78eb9650175\") " pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:49.254446 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.254399 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/601313cf-1384-4e54-ae22-00571f07c440-iptables-alerter-script\") pod \"iptables-alerter-nk5s5\" (UID: \"601313cf-1384-4e54-ae22-00571f07c440\") " pod="openshift-network-operator/iptables-alerter-nk5s5" Jul 06 20:05:49.254525 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.254486 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzj6n\" (UniqueName: \"kubernetes.io/projected/601313cf-1384-4e54-ae22-00571f07c440-kube-api-access-kzj6n\") pod \"iptables-alerter-nk5s5\" (UID: \"601313cf-1384-4e54-ae22-00571f07c440\") " pod="openshift-network-operator/iptables-alerter-nk5s5" Jul 06 20:05:49.255405 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255382 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2lt7\" (UniqueName: \"kubernetes.io/projected/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-kube-api-access-n2lt7\") pod \"network-metrics-daemon-ztl9r\" (UID: \"8aa84d62-1d3e-4c96-8f8b-d78eb9650175\") " pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:49.255543 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255414 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-var-lib-cni-multus\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.255543 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255436 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-var-lib-kubelet\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.255543 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255475 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-hostroot\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.255543 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255504 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kkr6\" (UniqueName: \"kubernetes.io/projected/95ac569b-2c0f-4574-8bb7-38498017e7a5-kube-api-access-8kkr6\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.255543 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255529 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.255820 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255552 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.255820 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255569 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-cnibin\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.255820 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255592 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-var-lib-cni-bin\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.255820 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255613 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-system-cni-dir\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.255820 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255629 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-os-release\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.255820 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255645 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-run-k8s-cni-cncf-io\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.255820 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255674 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-etc-kubernetes\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.255820 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255701 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-os-release\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.255820 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255744 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsrjc\" (UniqueName: \"kubernetes.io/projected/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-kube-api-access-fsrjc\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.255820 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255782 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-multus-cni-dir\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.255820 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255805 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-run-netns\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.255820 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255824 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-run-multus-certs\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.256486 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255897 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95ac569b-2c0f-4574-8bb7-38498017e7a5-cni-binary-copy\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.256486 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255933 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-multus-conf-dir\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.256486 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255960 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/601313cf-1384-4e54-ae22-00571f07c440-host-slash\") pod \"iptables-alerter-nk5s5\" (UID: \"601313cf-1384-4e54-ae22-00571f07c440\") " pod="openshift-network-operator/iptables-alerter-nk5s5" Jul 06 20:05:49.256486 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.255987 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.256486 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.256013 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-system-cni-dir\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.256486 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.256037 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-multus-socket-dir-parent\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.256486 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.256064 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-cnibin\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.256486 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.256110 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95ac569b-2c0f-4574-8bb7-38498017e7a5-multus-daemon-config\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.256486 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.256134 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-cni-binary-copy\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.256486 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.256467 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:49.256837 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.256543 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:05:49.258927 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.258910 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.260356 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.260325 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 20:05:49.260356 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.260343 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 20:05:49.260507 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.260368 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 20:05:49.260507 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.260398 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 20:05:49.260507 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.260326 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 20:05:49.261347 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.261331 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wm5jd" Jul 06 20:05:49.261662 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.261568 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 20:05:49.261662 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.261617 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2fsmw\"" Jul 06 20:05:49.261819 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.261805 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nlvmv\"" Jul 06 20:05:49.261854 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.261812 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 20:05:49.263853 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.263769 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 20:05:49.263853 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.263827 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mpdph" Jul 06 20:05:49.264673 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.264658 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 20:05:49.264774 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.264758 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 20:05:49.264934 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.264916 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 20:05:49.265017 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.264927 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2dt99\"" Jul 06 20:05:49.268350 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.268333 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 20:05:49.268542 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.268523 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 20:05:49.268623 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.268578 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wf7sp\"" Jul 06 20:05:49.268623 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.268594 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 20:05:49.268736 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.268580 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 20:05:49.269316 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.269299 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 20:05:49.269376 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.269358 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-glx27\"" Jul 06 20:05:49.269376 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.269362 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 20:05:49.290877 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.290861 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qtnjp" Jul 06 20:05:49.305259 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.305239 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qtnjp" Jul 06 20:05:49.342038 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.342022 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 20:05:49.356784 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.356757 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-etc-openvswitch\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.356872 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.356793 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-log-socket\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.356872 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.356809 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/36bc4e9d-a829-4f2a-906e-754c4cef2a5d-serviceca\") pod \"node-ca-wm5jd\" (UID: \"36bc4e9d-a829-4f2a-906e-754c4cef2a5d\") " pod="openshift-image-registry/node-ca-wm5jd" Jul 06 20:05:49.356872 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.356839 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-multus-cni-dir\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.357004 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.356868 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-run-netns\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.357004 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.356926 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-run-multus-certs\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.357004 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.356941 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-sysconfig\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.357004 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.356965 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9dvl\" (UniqueName: \"kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl\") pod \"network-check-target-spktr\" (UID: \"0e1abbba-697e-4907-999a-848e5c5ea695\") " pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:05:49.357004 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.356989 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-run-ovn\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.357183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.357016 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-multus-cni-dir\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.357183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.357016 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-run-multus-certs\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.357183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.357035 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95ac569b-2c0f-4574-8bb7-38498017e7a5-cni-binary-copy\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.357183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.357052 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-run-netns\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.357828 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.357809 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-multus-conf-dir\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.357867 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.357843 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/601313cf-1384-4e54-ae22-00571f07c440-host-slash\") pod \"iptables-alerter-nk5s5\" (UID: \"601313cf-1384-4e54-ae22-00571f07c440\") " pod="openshift-network-operator/iptables-alerter-nk5s5" Jul 06 20:05:49.357915 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.357872 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.357984 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.357912 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/601313cf-1384-4e54-ae22-00571f07c440-host-slash\") pod \"iptables-alerter-nk5s5\" (UID: \"601313cf-1384-4e54-ae22-00571f07c440\") " pod="openshift-network-operator/iptables-alerter-nk5s5" Jul 06 20:05:49.357984 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.357917 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-multus-conf-dir\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.357984 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.357952 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dngnq\" (UniqueName: \"kubernetes.io/projected/36bc4e9d-a829-4f2a-906e-754c4cef2a5d-kube-api-access-dngnq\") pod \"node-ca-wm5jd\" (UID: \"36bc4e9d-a829-4f2a-906e-754c4cef2a5d\") " pod="openshift-image-registry/node-ca-wm5jd" Jul 06 20:05:49.357984 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.357975 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-system-cni-dir\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.358186 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.357993 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-cnibin\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.358186 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358036 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-var-lib-kubelet\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.358186 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358059 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-kubelet\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.358186 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358079 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-systemd-units\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.358186 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358096 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-system-cni-dir\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.358186 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358102 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e23596bc-6c8a-4070-b549-eec43455f210-ovnkube-config\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.358186 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358135 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:49.358186 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358140 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-cnibin\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.358186 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358163 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fcc19e6-df9c-4444-981c-898f00976382-tmp-dir\") pod \"node-resolver-mpdph\" (UID: \"0fcc19e6-df9c-4444-981c-898f00976382\") " pod="openshift-dns/node-resolver-mpdph" Jul 06 20:05:49.358186 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358187 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-cni-binary-copy\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358207 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-run-systemd\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358244 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b98a719-eb47-4991-93e9-271e9a4f0307-kubelet-config\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358271 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/601313cf-1384-4e54-ae22-00571f07c440-iptables-alerter-script\") pod \"iptables-alerter-nk5s5\" (UID: \"601313cf-1384-4e54-ae22-00571f07c440\") " pod="openshift-network-operator/iptables-alerter-nk5s5" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358289 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzj6n\" (UniqueName: \"kubernetes.io/projected/601313cf-1384-4e54-ae22-00571f07c440-kube-api-access-kzj6n\") pod \"iptables-alerter-nk5s5\" (UID: \"601313cf-1384-4e54-ae22-00571f07c440\") " pod="openshift-network-operator/iptables-alerter-nk5s5" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358308 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-systemd\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358349 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-run-netns\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358395 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkv9h\" (UniqueName: \"kubernetes.io/projected/0fcc19e6-df9c-4444-981c-898f00976382-kube-api-access-kkv9h\") pod \"node-resolver-mpdph\" (UID: \"0fcc19e6-df9c-4444-981c-898f00976382\") " pod="openshift-dns/node-resolver-mpdph" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358423 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95ac569b-2c0f-4574-8bb7-38498017e7a5-cni-binary-copy\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358436 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kkr6\" (UniqueName: \"kubernetes.io/projected/95ac569b-2c0f-4574-8bb7-38498017e7a5-kube-api-access-8kkr6\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358423 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358465 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358507 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-run\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358560 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmhz2\" (UniqueName: \"kubernetes.io/projected/770ab8d5-830a-4afe-9ee9-f453bf43a381-kube-api-access-bmhz2\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358585 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/85293479-2c8e-4065-9ec5-7f5f44b853fb-agent-certs\") pod \"konnectivity-agent-wvqlx\" (UID: \"85293479-2c8e-4065-9ec5-7f5f44b853fb\") " pod="kube-system/konnectivity-agent-wvqlx" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358642 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.358712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358701 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-cni-binary-copy\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358685 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e23596bc-6c8a-4070-b549-eec43455f210-ovnkube-script-lib\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358747 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-cnibin\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358794 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-system-cni-dir\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358825 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-cnibin\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358897 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-etc-selinux\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358912 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/601313cf-1384-4e54-ae22-00571f07c440-iptables-alerter-script\") pod \"iptables-alerter-nk5s5\" (UID: \"601313cf-1384-4e54-ae22-00571f07c440\") " pod="openshift-network-operator/iptables-alerter-nk5s5" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358935 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-system-cni-dir\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358972 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-sys-fs\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.358991 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b98a719-eb47-4991-93e9-271e9a4f0307-dbus\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359013 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0fcc19e6-df9c-4444-981c-898f00976382-hosts-file\") pod \"node-resolver-mpdph\" (UID: \"0fcc19e6-df9c-4444-981c-898f00976382\") " pod="openshift-dns/node-resolver-mpdph" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359013 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359031 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-os-release\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359066 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsrjc\" (UniqueName: \"kubernetes.io/projected/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-kube-api-access-fsrjc\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359126 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-kubernetes\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359137 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-os-release\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359235 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-modprobe-d\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.359493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359272 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-lib-modules\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359308 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-host\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359334 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359382 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-multus-socket-dir-parent\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359407 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-tuned\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359431 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwjdv\" (UniqueName: \"kubernetes.io/projected/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-kube-api-access-qwjdv\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359455 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-multus-socket-dir-parent\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359464 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-device-dir\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359491 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36bc4e9d-a829-4f2a-906e-754c4cef2a5d-host\") pod \"node-ca-wm5jd\" (UID: \"36bc4e9d-a829-4f2a-906e-754c4cef2a5d\") " pod="openshift-image-registry/node-ca-wm5jd" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359515 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95ac569b-2c0f-4574-8bb7-38498017e7a5-multus-daemon-config\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359546 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-sysctl-conf\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359571 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-node-log\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359598 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-697kb\" (UniqueName: \"kubernetes.io/projected/e23596bc-6c8a-4070-b549-eec43455f210-kube-api-access-697kb\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359632 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs\") pod \"network-metrics-daemon-ztl9r\" (UID: \"8aa84d62-1d3e-4c96-8f8b-d78eb9650175\") " pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359657 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-sys\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359680 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-tmp\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.359740 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:05:49.360313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359768 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/85293479-2c8e-4065-9ec5-7f5f44b853fb-konnectivity-ca\") pod \"konnectivity-agent-wvqlx\" (UID: \"85293479-2c8e-4065-9ec5-7f5f44b853fb\") " pod="kube-system/konnectivity-agent-wvqlx" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359795 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-run-openvswitch\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359813 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-cni-bin\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.359832 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs podName:8aa84d62-1d3e-4c96-8f8b-d78eb9650175 nodeName:}" failed. No retries permitted until 2026-07-06 20:05:49.859810375 +0000 UTC m=+2.986722459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs") pod "network-metrics-daemon-ztl9r" (UID: "8aa84d62-1d3e-4c96-8f8b-d78eb9650175") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359866 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e23596bc-6c8a-4070-b549-eec43455f210-env-overrides\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359910 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2lt7\" (UniqueName: \"kubernetes.io/projected/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-kube-api-access-n2lt7\") pod \"network-metrics-daemon-ztl9r\" (UID: \"8aa84d62-1d3e-4c96-8f8b-d78eb9650175\") " pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359926 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95ac569b-2c0f-4574-8bb7-38498017e7a5-multus-daemon-config\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359951 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-var-lib-cni-multus\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.359998 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-var-lib-cni-multus\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360015 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-var-lib-kubelet\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360049 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-hostroot\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360074 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-var-lib-kubelet\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360075 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360108 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-hostroot\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360131 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-sysctl-d\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360167 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-cni-netd\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.361058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360198 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e23596bc-6c8a-4070-b549-eec43455f210-ovn-node-metrics-cert\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.361858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360223 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-var-lib-cni-bin\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.361858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360250 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-socket-dir\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.361858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360261 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.361858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360277 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-registration-dir\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.361858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360266 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-var-lib-cni-bin\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.361858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360607 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-slash\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.361858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360656 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-var-lib-openvswitch\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.361858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360699 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-run-ovn-kubernetes\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.361858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360741 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-os-release\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.361858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360815 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-run-k8s-cni-cncf-io\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.361858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360856 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-host-run-k8s-cni-cncf-io\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.361858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360863 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-etc-kubernetes\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.361858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.360954 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-os-release\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.361858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.361035 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95ac569b-2c0f-4574-8bb7-38498017e7a5-etc-kubernetes\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.371032 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.371011 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 20:05:49.374427 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.374407 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzj6n\" (UniqueName: \"kubernetes.io/projected/601313cf-1384-4e54-ae22-00571f07c440-kube-api-access-kzj6n\") pod \"iptables-alerter-nk5s5\" (UID: \"601313cf-1384-4e54-ae22-00571f07c440\") " pod="openshift-network-operator/iptables-alerter-nk5s5" Jul 06 20:05:49.374503 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.374409 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsrjc\" (UniqueName: \"kubernetes.io/projected/b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4-kube-api-access-fsrjc\") pod \"multus-additional-cni-plugins-8jp5d\" (UID: \"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4\") " pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.377623 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.377602 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kkr6\" (UniqueName: \"kubernetes.io/projected/95ac569b-2c0f-4574-8bb7-38498017e7a5-kube-api-access-8kkr6\") pod \"multus-7t42j\" (UID: \"95ac569b-2c0f-4574-8bb7-38498017e7a5\") " pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.379061 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.379036 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2lt7\" (UniqueName: \"kubernetes.io/projected/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-kube-api-access-n2lt7\") pod \"network-metrics-daemon-ztl9r\" (UID: \"8aa84d62-1d3e-4c96-8f8b-d78eb9650175\") " pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:49.462462 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462409 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwjdv\" (UniqueName: \"kubernetes.io/projected/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-kube-api-access-qwjdv\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.462462 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462436 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-device-dir\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.462462 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462454 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36bc4e9d-a829-4f2a-906e-754c4cef2a5d-host\") pod \"node-ca-wm5jd\" (UID: \"36bc4e9d-a829-4f2a-906e-754c4cef2a5d\") " pod="openshift-image-registry/node-ca-wm5jd" Jul 06 20:05:49.462648 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462471 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-sysctl-conf\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.462648 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462491 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-node-log\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.462648 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462516 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-697kb\" (UniqueName: \"kubernetes.io/projected/e23596bc-6c8a-4070-b549-eec43455f210-kube-api-access-697kb\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.462648 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462532 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-device-dir\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.462648 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462552 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-sys\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.462648 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462553 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/36bc4e9d-a829-4f2a-906e-754c4cef2a5d-host\") pod \"node-ca-wm5jd\" (UID: \"36bc4e9d-a829-4f2a-906e-754c4cef2a5d\") " pod="openshift-image-registry/node-ca-wm5jd" Jul 06 20:05:49.462648 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462604 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-sys\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.462648 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462600 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-node-log\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.462648 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462608 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-tmp\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.462648 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462643 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-sysctl-conf\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462672 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/85293479-2c8e-4065-9ec5-7f5f44b853fb-konnectivity-ca\") pod \"konnectivity-agent-wvqlx\" (UID: \"85293479-2c8e-4065-9ec5-7f5f44b853fb\") " pod="kube-system/konnectivity-agent-wvqlx" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462704 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-run-openvswitch\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462730 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-cni-bin\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462756 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e23596bc-6c8a-4070-b549-eec43455f210-env-overrides\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462787 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-sysctl-d\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462796 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-cni-bin\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462758 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-run-openvswitch\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462812 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-cni-netd\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462864 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e23596bc-6c8a-4070-b549-eec43455f210-ovn-node-metrics-cert\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462896 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-cni-netd\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462910 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-socket-dir\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462929 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-sysctl-d\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.462987 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-registration-dir\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463023 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-slash\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463043 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-socket-dir\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463052 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-var-lib-openvswitch\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463079 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-registration-dir\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463095 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-var-lib-openvswitch\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463086 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-run-ovn-kubernetes\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463121 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-run-ovn-kubernetes\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463140 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-slash\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463141 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-etc-openvswitch\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463173 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-etc-openvswitch\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463177 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-log-socket\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463209 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/36bc4e9d-a829-4f2a-906e-754c4cef2a5d-serviceca\") pod \"node-ca-wm5jd\" (UID: \"36bc4e9d-a829-4f2a-906e-754c4cef2a5d\") " pod="openshift-image-registry/node-ca-wm5jd" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463240 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-sysconfig\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463212 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-log-socket\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463250 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e23596bc-6c8a-4070-b549-eec43455f210-env-overrides\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463268 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9dvl\" (UniqueName: \"kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl\") pod \"network-check-target-spktr\" (UID: \"0e1abbba-697e-4907-999a-848e5c5ea695\") " pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463239 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/85293479-2c8e-4065-9ec5-7f5f44b853fb-konnectivity-ca\") pod \"konnectivity-agent-wvqlx\" (UID: \"85293479-2c8e-4065-9ec5-7f5f44b853fb\") " pod="kube-system/konnectivity-agent-wvqlx" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463296 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-run-ovn\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463303 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-sysconfig\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463325 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dngnq\" (UniqueName: \"kubernetes.io/projected/36bc4e9d-a829-4f2a-906e-754c4cef2a5d-kube-api-access-dngnq\") pod \"node-ca-wm5jd\" (UID: \"36bc4e9d-a829-4f2a-906e-754c4cef2a5d\") " pod="openshift-image-registry/node-ca-wm5jd" Jul 06 20:05:49.463996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463347 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-run-ovn\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463354 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-var-lib-kubelet\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463397 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-kubelet\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463415 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-systemd-units\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463435 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e23596bc-6c8a-4070-b549-eec43455f210-ovnkube-config\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463460 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-kubelet\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463462 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463494 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-var-lib-kubelet\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463498 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fcc19e6-df9c-4444-981c-898f00976382-tmp-dir\") pod \"node-resolver-mpdph\" (UID: \"0fcc19e6-df9c-4444-981c-898f00976382\") " pod="openshift-dns/node-resolver-mpdph" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463525 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-run-systemd\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463532 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-systemd-units\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.463539 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463556 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b98a719-eb47-4991-93e9-271e9a4f0307-kubelet-config\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.463592 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret podName:0b98a719-eb47-4991-93e9-271e9a4f0307 nodeName:}" failed. No retries permitted until 2026-07-06 20:05:49.96357303 +0000 UTC m=+3.090485129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret") pod "global-pull-secret-syncer-r6ptl" (UID: "0b98a719-eb47-4991-93e9-271e9a4f0307") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463598 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b98a719-eb47-4991-93e9-271e9a4f0307-kubelet-config\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463609 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-systemd\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463636 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-run-netns\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.464864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463635 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/36bc4e9d-a829-4f2a-906e-754c4cef2a5d-serviceca\") pod \"node-ca-wm5jd\" (UID: \"36bc4e9d-a829-4f2a-906e-754c4cef2a5d\") " pod="openshift-image-registry/node-ca-wm5jd" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463661 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkv9h\" (UniqueName: \"kubernetes.io/projected/0fcc19e6-df9c-4444-981c-898f00976382-kube-api-access-kkv9h\") pod \"node-resolver-mpdph\" (UID: \"0fcc19e6-df9c-4444-981c-898f00976382\") " pod="openshift-dns/node-resolver-mpdph" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463699 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-run\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463714 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-systemd\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463724 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmhz2\" (UniqueName: \"kubernetes.io/projected/770ab8d5-830a-4afe-9ee9-f453bf43a381-kube-api-access-bmhz2\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463754 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/85293479-2c8e-4065-9ec5-7f5f44b853fb-agent-certs\") pod \"konnectivity-agent-wvqlx\" (UID: \"85293479-2c8e-4065-9ec5-7f5f44b853fb\") " pod="kube-system/konnectivity-agent-wvqlx" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463767 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-run-netns\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463776 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463795 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e23596bc-6c8a-4070-b549-eec43455f210-ovnkube-script-lib\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463818 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-etc-selinux\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463820 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fcc19e6-df9c-4444-981c-898f00976382-tmp-dir\") pod \"node-resolver-mpdph\" (UID: \"0fcc19e6-df9c-4444-981c-898f00976382\") " pod="openshift-dns/node-resolver-mpdph" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463839 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-sys-fs\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463857 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b98a719-eb47-4991-93e9-271e9a4f0307-dbus\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463873 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0fcc19e6-df9c-4444-981c-898f00976382-hosts-file\") pod \"node-resolver-mpdph\" (UID: \"0fcc19e6-df9c-4444-981c-898f00976382\") " pod="openshift-dns/node-resolver-mpdph" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463891 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-kubernetes\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463902 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-run-systemd\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463928 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-modprobe-d\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.465770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463963 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-lib-modules\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463972 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-etc-selinux\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463978 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-host\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.463972 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e23596bc-6c8a-4070-b549-eec43455f210-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.464027 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-host\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.464048 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-run\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.464046 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.464076 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e23596bc-6c8a-4070-b549-eec43455f210-ovnkube-config\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.464084 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-sys-fs\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.464104 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/770ab8d5-830a-4afe-9ee9-f453bf43a381-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.464116 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-tuned\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.464153 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b98a719-eb47-4991-93e9-271e9a4f0307-dbus\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.464196 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-lib-modules\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.464211 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-modprobe-d\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.464228 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0fcc19e6-df9c-4444-981c-898f00976382-hosts-file\") pod \"node-resolver-mpdph\" (UID: \"0fcc19e6-df9c-4444-981c-898f00976382\") " pod="openshift-dns/node-resolver-mpdph" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.464272 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-kubernetes\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.464607 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e23596bc-6c8a-4070-b549-eec43455f210-ovnkube-script-lib\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.465168 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-tmp\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.466501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.466068 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/85293479-2c8e-4065-9ec5-7f5f44b853fb-agent-certs\") pod \"konnectivity-agent-wvqlx\" (UID: \"85293479-2c8e-4065-9ec5-7f5f44b853fb\") " pod="kube-system/konnectivity-agent-wvqlx" Jul 06 20:05:49.467052 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.466151 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-etc-tuned\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.467052 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.466448 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e23596bc-6c8a-4070-b549-eec43455f210-ovn-node-metrics-cert\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.481900 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.481879 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwjdv\" (UniqueName: \"kubernetes.io/projected/f8f276d9-9aed-42a3-8e16-04d4a5e5eba5-kube-api-access-qwjdv\") pod \"tuned-65r94\" (UID: \"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5\") " pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.482756 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.482716 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal" event={"ID":"19411eae83bc603ba88b7f2105ff736e","Type":"ContainerStarted","Data":"3f36f91999d06c0fa426f624e7bc050a057b5900b7d34a110ad15ce36ac94586"} Jul 06 20:05:49.483857 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.483837 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" event={"ID":"d927aedbafdf82c88fcd9938008cb116","Type":"ContainerStarted","Data":"b80cf11967ac48364cffbb3b2f7034efa5ea1e647d81a5ca7da168beab73d60c"} Jul 06 20:05:49.493984 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.493967 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:05:49.499354 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.499339 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmhz2\" (UniqueName: \"kubernetes.io/projected/770ab8d5-830a-4afe-9ee9-f453bf43a381-kube-api-access-bmhz2\") pod \"aws-ebs-csi-driver-node-m6qc4\" (UID: \"770ab8d5-830a-4afe-9ee9-f453bf43a381\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.500310 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.500295 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:05:49.500369 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.500319 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:05:49.500369 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.500330 2583 projected.go:196] Error preparing data for projected volume kube-api-access-h9dvl for pod openshift-network-diagnostics/network-check-target-spktr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:05:49.500435 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.500392 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl podName:0e1abbba-697e-4907-999a-848e5c5ea695 nodeName:}" failed. No retries permitted until 2026-07-06 20:05:50.000374258 +0000 UTC m=+3.127286360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h9dvl" (UniqueName: "kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl") pod "network-check-target-spktr" (UID: "0e1abbba-697e-4907-999a-848e5c5ea695") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:05:49.504010 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.503995 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-697kb\" (UniqueName: \"kubernetes.io/projected/e23596bc-6c8a-4070-b549-eec43455f210-kube-api-access-697kb\") pod \"ovnkube-node-4m62r\" (UID: \"e23596bc-6c8a-4070-b549-eec43455f210\") " pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.519543 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.519523 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkv9h\" (UniqueName: \"kubernetes.io/projected/0fcc19e6-df9c-4444-981c-898f00976382-kube-api-access-kkv9h\") pod \"node-resolver-mpdph\" (UID: \"0fcc19e6-df9c-4444-981c-898f00976382\") " pod="openshift-dns/node-resolver-mpdph" Jul 06 20:05:49.519735 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.519719 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dngnq\" (UniqueName: \"kubernetes.io/projected/36bc4e9d-a829-4f2a-906e-754c4cef2a5d-kube-api-access-dngnq\") pod \"node-ca-wm5jd\" (UID: \"36bc4e9d-a829-4f2a-906e-754c4cef2a5d\") " pod="openshift-image-registry/node-ca-wm5jd" Jul 06 20:05:49.545159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.545139 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nk5s5" Jul 06 20:05:49.555081 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.555062 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8jp5d" Jul 06 20:05:49.555392 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:49.555363 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod601313cf_1384_4e54_ae22_00571f07c440.slice/crio-878416b450e3facb2a47b04f9386c0afcc1fd0b03e5b4dc7079b4a73a30480d4 WatchSource:0}: Error finding container 878416b450e3facb2a47b04f9386c0afcc1fd0b03e5b4dc7079b4a73a30480d4: Status 404 returned error can't find the container with id 878416b450e3facb2a47b04f9386c0afcc1fd0b03e5b4dc7079b4a73a30480d4 Jul 06 20:05:49.561778 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.561761 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7t42j" Jul 06 20:05:49.563586 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:49.563564 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9369d9b_5e6e_4ff9_9760_b79c93fb5cd4.slice/crio-063441964bad6b3dcf11290dacd7824d83ebf35270fcf77602f8fb86524ec967 WatchSource:0}: Error finding container 063441964bad6b3dcf11290dacd7824d83ebf35270fcf77602f8fb86524ec967: Status 404 returned error can't find the container with id 063441964bad6b3dcf11290dacd7824d83ebf35270fcf77602f8fb86524ec967 Jul 06 20:05:49.567648 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.567629 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-65r94" Jul 06 20:05:49.570931 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:49.570911 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95ac569b_2c0f_4574_8bb7_38498017e7a5.slice/crio-10faa4e89b8b5102c8f934783373e3b2a46b3c6e188c801e95bfd12c91fbc9d4 WatchSource:0}: Error finding container 10faa4e89b8b5102c8f934783373e3b2a46b3c6e188c801e95bfd12c91fbc9d4: Status 404 returned error can't find the container with id 10faa4e89b8b5102c8f934783373e3b2a46b3c6e188c801e95bfd12c91fbc9d4 Jul 06 20:05:49.574690 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.572342 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wvqlx" Jul 06 20:05:49.578116 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:49.577104 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f276d9_9aed_42a3_8e16_04d4a5e5eba5.slice/crio-8f2ee7e15c60b0a20a6308b5c155058b08306a59c09121fb33a14df5a6a0a752 WatchSource:0}: Error finding container 8f2ee7e15c60b0a20a6308b5c155058b08306a59c09121fb33a14df5a6a0a752: Status 404 returned error can't find the container with id 8f2ee7e15c60b0a20a6308b5c155058b08306a59c09121fb33a14df5a6a0a752 Jul 06 20:05:49.578116 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.578112 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:05:49.584097 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:49.584078 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85293479_2c8e_4065_9ec5_7f5f44b853fb.slice/crio-5569d8153d986bd24f0977a77c8095cd5c28dc8db8100aad4e916f894b14536d WatchSource:0}: Error finding container 5569d8153d986bd24f0977a77c8095cd5c28dc8db8100aad4e916f894b14536d: Status 404 returned error can't find the container with id 5569d8153d986bd24f0977a77c8095cd5c28dc8db8100aad4e916f894b14536d Jul 06 20:05:49.584964 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.584941 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" Jul 06 20:05:49.587689 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:49.587655 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode23596bc_6c8a_4070_b549_eec43455f210.slice/crio-9548e50bede10fe339110fa6f6339123e6300d183a3ddc3ba1e470422b834c72 WatchSource:0}: Error finding container 9548e50bede10fe339110fa6f6339123e6300d183a3ddc3ba1e470422b834c72: Status 404 returned error can't find the container with id 9548e50bede10fe339110fa6f6339123e6300d183a3ddc3ba1e470422b834c72 Jul 06 20:05:49.589995 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.589948 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wm5jd" Jul 06 20:05:49.594836 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:49.594816 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod770ab8d5_830a_4afe_9ee9_f453bf43a381.slice/crio-55a52f94295bcdb6b1d4b0182900ff6dc3de2c150963957850bfde69b23dbf21 WatchSource:0}: Error finding container 55a52f94295bcdb6b1d4b0182900ff6dc3de2c150963957850bfde69b23dbf21: Status 404 returned error can't find the container with id 55a52f94295bcdb6b1d4b0182900ff6dc3de2c150963957850bfde69b23dbf21 Jul 06 20:05:49.595530 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.595514 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mpdph" Jul 06 20:05:49.599383 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:49.599277 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36bc4e9d_a829_4f2a_906e_754c4cef2a5d.slice/crio-bf58b6a90cd1c47a121574f2f5b5d505ca6e0b9dbe947b850c0e235c990fb5e8 WatchSource:0}: Error finding container bf58b6a90cd1c47a121574f2f5b5d505ca6e0b9dbe947b850c0e235c990fb5e8: Status 404 returned error can't find the container with id bf58b6a90cd1c47a121574f2f5b5d505ca6e0b9dbe947b850c0e235c990fb5e8 Jul 06 20:05:49.605407 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:05:49.605389 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fcc19e6_df9c_4444_981c_898f00976382.slice/crio-b67aa181269561c1844440276f02165b30c8bb5666195d38e37a828bb3366a55 WatchSource:0}: Error finding container b67aa181269561c1844440276f02165b30c8bb5666195d38e37a828bb3366a55: Status 404 returned error can't find the container with id b67aa181269561c1844440276f02165b30c8bb5666195d38e37a828bb3366a55 Jul 06 20:05:49.866416 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.866343 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs\") pod \"network-metrics-daemon-ztl9r\" (UID: \"8aa84d62-1d3e-4c96-8f8b-d78eb9650175\") " pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:49.866552 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.866486 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:05:49.866552 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.866550 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs podName:8aa84d62-1d3e-4c96-8f8b-d78eb9650175 nodeName:}" failed. No retries permitted until 2026-07-06 20:05:50.86652945 +0000 UTC m=+3.993441553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs") pod "network-metrics-daemon-ztl9r" (UID: "8aa84d62-1d3e-4c96-8f8b-d78eb9650175") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:05:49.967532 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:49.967495 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:49.967701 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.967637 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:05:49.967762 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:49.967702 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret podName:0b98a719-eb47-4991-93e9-271e9a4f0307 nodeName:}" failed. No retries permitted until 2026-07-06 20:05:50.967682274 +0000 UTC m=+4.094594361 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret") pod "global-pull-secret-syncer-r6ptl" (UID: "0b98a719-eb47-4991-93e9-271e9a4f0307") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:05:50.068328 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.068289 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9dvl\" (UniqueName: \"kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl\") pod \"network-check-target-spktr\" (UID: \"0e1abbba-697e-4907-999a-848e5c5ea695\") " pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:05:50.068470 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:50.068445 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:05:50.068470 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:50.068465 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:05:50.068605 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:50.068478 2583 projected.go:196] Error preparing data for projected volume kube-api-access-h9dvl for pod openshift-network-diagnostics/network-check-target-spktr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:05:50.068605 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:50.068541 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl podName:0e1abbba-697e-4907-999a-848e5c5ea695 nodeName:}" failed. No retries permitted until 2026-07-06 20:05:51.068520974 +0000 UTC m=+4.195433066 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-h9dvl" (UniqueName: "kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl") pod "network-check-target-spktr" (UID: "0e1abbba-697e-4907-999a-848e5c5ea695") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:05:50.308783 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.308647 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 20:00:49 +0000 UTC" deadline="2028-01-30 01:56:01.424390884 +0000 UTC" Jul 06 20:05:50.308783 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.308692 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13733h50m11.115701141s" Jul 06 20:05:50.515970 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.515917 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" event={"ID":"770ab8d5-830a-4afe-9ee9-f453bf43a381","Type":"ContainerStarted","Data":"55a52f94295bcdb6b1d4b0182900ff6dc3de2c150963957850bfde69b23dbf21"} Jul 06 20:05:50.530451 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.530417 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 20:05:50.534631 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.534591 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wvqlx" event={"ID":"85293479-2c8e-4065-9ec5-7f5f44b853fb","Type":"ContainerStarted","Data":"5569d8153d986bd24f0977a77c8095cd5c28dc8db8100aad4e916f894b14536d"} Jul 06 20:05:50.555408 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.555376 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7t42j" event={"ID":"95ac569b-2c0f-4574-8bb7-38498017e7a5","Type":"ContainerStarted","Data":"10faa4e89b8b5102c8f934783373e3b2a46b3c6e188c801e95bfd12c91fbc9d4"} Jul 06 20:05:50.560092 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.559997 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jp5d" event={"ID":"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4","Type":"ContainerStarted","Data":"063441964bad6b3dcf11290dacd7824d83ebf35270fcf77602f8fb86524ec967"} Jul 06 20:05:50.587649 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.587567 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nk5s5" event={"ID":"601313cf-1384-4e54-ae22-00571f07c440","Type":"ContainerStarted","Data":"878416b450e3facb2a47b04f9386c0afcc1fd0b03e5b4dc7079b4a73a30480d4"} Jul 06 20:05:50.595052 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.594913 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mpdph" event={"ID":"0fcc19e6-df9c-4444-981c-898f00976382","Type":"ContainerStarted","Data":"b67aa181269561c1844440276f02165b30c8bb5666195d38e37a828bb3366a55"} Jul 06 20:05:50.610179 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.608658 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wm5jd" event={"ID":"36bc4e9d-a829-4f2a-906e-754c4cef2a5d","Type":"ContainerStarted","Data":"bf58b6a90cd1c47a121574f2f5b5d505ca6e0b9dbe947b850c0e235c990fb5e8"} Jul 06 20:05:50.616655 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.616629 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" event={"ID":"e23596bc-6c8a-4070-b549-eec43455f210","Type":"ContainerStarted","Data":"9548e50bede10fe339110fa6f6339123e6300d183a3ddc3ba1e470422b834c72"} Jul 06 20:05:50.641725 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.641649 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-65r94" event={"ID":"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5","Type":"ContainerStarted","Data":"8f2ee7e15c60b0a20a6308b5c155058b08306a59c09121fb33a14df5a6a0a752"} Jul 06 20:05:50.876387 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.875772 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs\") pod \"network-metrics-daemon-ztl9r\" (UID: \"8aa84d62-1d3e-4c96-8f8b-d78eb9650175\") " pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:50.876387 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:50.875932 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:05:50.876387 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:50.875996 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs podName:8aa84d62-1d3e-4c96-8f8b-d78eb9650175 nodeName:}" failed. No retries permitted until 2026-07-06 20:05:52.875975881 +0000 UTC m=+6.002887987 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs") pod "network-metrics-daemon-ztl9r" (UID: "8aa84d62-1d3e-4c96-8f8b-d78eb9650175") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:05:50.977141 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:50.976631 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:50.977141 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:50.976769 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:05:50.977141 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:50.976824 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret podName:0b98a719-eb47-4991-93e9-271e9a4f0307 nodeName:}" failed. No retries permitted until 2026-07-06 20:05:52.976806645 +0000 UTC m=+6.103718732 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret") pod "global-pull-secret-syncer-r6ptl" (UID: "0b98a719-eb47-4991-93e9-271e9a4f0307") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:05:51.077638 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:51.077601 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9dvl\" (UniqueName: \"kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl\") pod \"network-check-target-spktr\" (UID: \"0e1abbba-697e-4907-999a-848e5c5ea695\") " pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:05:51.077803 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:51.077750 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:05:51.077803 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:51.077767 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:05:51.077803 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:51.077778 2583 projected.go:196] Error preparing data for projected volume kube-api-access-h9dvl for pod openshift-network-diagnostics/network-check-target-spktr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:05:51.077967 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:51.077832 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl podName:0e1abbba-697e-4907-999a-848e5c5ea695 nodeName:}" failed. No retries permitted until 2026-07-06 20:05:53.077814404 +0000 UTC m=+6.204726504 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-h9dvl" (UniqueName: "kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl") pod "network-check-target-spktr" (UID: "0e1abbba-697e-4907-999a-848e5c5ea695") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:05:51.310046 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:51.309819 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 20:00:49 +0000 UTC" deadline="2028-01-26 11:05:00.02713342 +0000 UTC" Jul 06 20:05:51.310046 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:51.309856 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13646h59m8.717281531s" Jul 06 20:05:51.479200 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:51.479172 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:51.479349 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:51.479171 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:05:51.479349 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:51.479309 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:51.479349 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:51.479307 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:05:51.479532 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:51.479410 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:05:51.479532 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:51.479500 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:05:52.893828 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:52.893792 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs\") pod \"network-metrics-daemon-ztl9r\" (UID: \"8aa84d62-1d3e-4c96-8f8b-d78eb9650175\") " pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:52.894292 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:52.893999 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:05:52.894292 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:52.894063 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs podName:8aa84d62-1d3e-4c96-8f8b-d78eb9650175 nodeName:}" failed. No retries permitted until 2026-07-06 20:05:56.894043706 +0000 UTC m=+10.020955806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs") pod "network-metrics-daemon-ztl9r" (UID: "8aa84d62-1d3e-4c96-8f8b-d78eb9650175") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:05:52.995039 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:52.995005 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:52.995219 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:52.995150 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:05:52.995287 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:52.995221 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret podName:0b98a719-eb47-4991-93e9-271e9a4f0307 nodeName:}" failed. No retries permitted until 2026-07-06 20:05:56.995199687 +0000 UTC m=+10.122111770 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret") pod "global-pull-secret-syncer-r6ptl" (UID: "0b98a719-eb47-4991-93e9-271e9a4f0307") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:05:53.096333 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:53.096298 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9dvl\" (UniqueName: \"kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl\") pod \"network-check-target-spktr\" (UID: \"0e1abbba-697e-4907-999a-848e5c5ea695\") " pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:05:53.096498 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:53.096453 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:05:53.096498 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:53.096478 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:05:53.096498 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:53.096491 2583 projected.go:196] Error preparing data for projected volume kube-api-access-h9dvl for pod openshift-network-diagnostics/network-check-target-spktr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:05:53.096706 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:53.096574 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl podName:0e1abbba-697e-4907-999a-848e5c5ea695 nodeName:}" failed. No retries permitted until 2026-07-06 20:05:57.096553025 +0000 UTC m=+10.223465112 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-h9dvl" (UniqueName: "kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl") pod "network-check-target-spktr" (UID: "0e1abbba-697e-4907-999a-848e5c5ea695") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:05:53.480015 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:53.479979 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:53.480196 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:53.480117 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:05:53.480519 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:53.480497 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:53.480623 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:53.480603 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:05:53.480689 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:53.480677 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:05:53.480766 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:53.480747 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:05:55.481125 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:55.481062 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:55.481529 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:55.481156 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:05:55.481529 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:55.481455 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:05:55.481636 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:55.481531 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:05:55.481636 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:55.481580 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:55.481742 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:55.481651 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:05:56.925000 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:56.924962 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs\") pod \"network-metrics-daemon-ztl9r\" (UID: \"8aa84d62-1d3e-4c96-8f8b-d78eb9650175\") " pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:56.925438 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:56.925151 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:05:56.925438 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:56.925211 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs podName:8aa84d62-1d3e-4c96-8f8b-d78eb9650175 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:04.925190362 +0000 UTC m=+18.052102445 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs") pod "network-metrics-daemon-ztl9r" (UID: "8aa84d62-1d3e-4c96-8f8b-d78eb9650175") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:05:57.026253 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:57.026221 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:57.026420 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:57.026357 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:05:57.026420 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:57.026418 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret podName:0b98a719-eb47-4991-93e9-271e9a4f0307 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:05.026401524 +0000 UTC m=+18.153313620 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret") pod "global-pull-secret-syncer-r6ptl" (UID: "0b98a719-eb47-4991-93e9-271e9a4f0307") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:05:57.127135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:57.127104 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9dvl\" (UniqueName: \"kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl\") pod \"network-check-target-spktr\" (UID: \"0e1abbba-697e-4907-999a-848e5c5ea695\") " pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:05:57.127431 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:57.127297 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:05:57.127431 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:57.127328 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:05:57.127431 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:57.127353 2583 projected.go:196] Error preparing data for projected volume kube-api-access-h9dvl for pod openshift-network-diagnostics/network-check-target-spktr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:05:57.127431 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:57.127415 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl podName:0e1abbba-697e-4907-999a-848e5c5ea695 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:05.127396891 +0000 UTC m=+18.254308995 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-h9dvl" (UniqueName: "kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl") pod "network-check-target-spktr" (UID: "0e1abbba-697e-4907-999a-848e5c5ea695") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:05:57.480210 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:57.480100 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:57.480210 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:57.480195 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:57.480426 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:57.480213 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:05:57.480426 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:57.480314 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:05:57.480426 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:57.480369 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:05:57.480426 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:57.480417 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:05:59.479121 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:59.479088 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:05:59.479572 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:59.479091 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:05:59.479572 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:59.479214 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:05:59.479572 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:59.479280 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:05:59.479572 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:05:59.479088 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:05:59.479572 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:05:59.479374 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:06:01.478939 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:01.478696 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:01.479386 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:01.479039 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:06:01.479386 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:01.478769 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:01.479386 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:01.478739 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:01.479386 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:01.479137 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:06:01.479386 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:01.479218 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:06:01.680698 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:01.680528 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal" event={"ID":"19411eae83bc603ba88b7f2105ff736e","Type":"ContainerStarted","Data":"a8afcb21198844af06af7fa520e23a3e36d29bc4aa120a48dd2cd61c8547757d"} Jul 06 20:06:01.700887 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:01.700816 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal" podStartSLOduration=13.700803921 podStartE2EDuration="13.700803921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:05:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:01.700728915 +0000 UTC m=+14.827641020" watchObservedRunningTime="2026-07-06 20:06:01.700803921 +0000 UTC m=+14.827716025" Jul 06 20:06:02.684918 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:02.684888 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" event={"ID":"770ab8d5-830a-4afe-9ee9-f453bf43a381","Type":"ContainerStarted","Data":"71e2deb708bec04dc908e3e78d63047482f3257e19d17c0c5a0fd56cb00c4f47"} Jul 06 20:06:02.686736 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:02.686710 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wvqlx" event={"ID":"85293479-2c8e-4065-9ec5-7f5f44b853fb","Type":"ContainerStarted","Data":"45d982b3e5fb0bbcc693b0811fd13c6ab952c1c287ac640babe63f3994c7a0bf"} Jul 06 20:06:02.688379 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:02.688352 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jp5d" event={"ID":"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4","Type":"ContainerStarted","Data":"85edde0ee68fde6ce11aed5e9b58694a6e308f8e0571f3c1df966c7a30618e7e"} Jul 06 20:06:02.689905 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:02.689865 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nk5s5" event={"ID":"601313cf-1384-4e54-ae22-00571f07c440","Type":"ContainerStarted","Data":"94ab96c0af33f91dd21572285313faa373f9600a6ec2207c9fe5e5fdc647a450"} Jul 06 20:06:02.691380 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:02.691360 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" event={"ID":"d927aedbafdf82c88fcd9938008cb116","Type":"ContainerStarted","Data":"2a5ac593305624a2724500eb074ca5fca134548d79eca78d20ee5a69a6f97e1c"} Jul 06 20:06:02.692901 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:02.692860 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mpdph" event={"ID":"0fcc19e6-df9c-4444-981c-898f00976382","Type":"ContainerStarted","Data":"ef62887b6ede501cbe88e2a2fcaea9a993d798da842e0244bca6ec80484908f6"} Jul 06 20:06:02.694375 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:02.694352 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wm5jd" event={"ID":"36bc4e9d-a829-4f2a-906e-754c4cef2a5d","Type":"ContainerStarted","Data":"d0388198ab7c6e5218e419d893fc193acbc261ba78894c674feb562fbce18986"} Jul 06 20:06:02.707328 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:02.707279 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wvqlx" podStartSLOduration=4.611878841 podStartE2EDuration="15.707263188s" totalImagesPullingTime="11.095384347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:02.707054768 +0000 UTC m=+15.833966875" watchObservedRunningTime="2026-07-06 20:06:02.707263188 +0000 UTC m=+15.834175293" Jul 06 20:06:02.743678 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:02.743632 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nk5s5" podStartSLOduration=4.619458765 podStartE2EDuration="15.74361517s" totalImagesPullingTime="11.124156405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:02.742826624 +0000 UTC m=+15.869738733" watchObservedRunningTime="2026-07-06 20:06:02.74361517 +0000 UTC m=+15.870527279" Jul 06 20:06:02.786790 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:02.786715 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wm5jd" podStartSLOduration=3.708612111 podStartE2EDuration="14.786698723s" totalImagesPullingTime="11.078086612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:02.785784563 +0000 UTC m=+15.912696671" watchObservedRunningTime="2026-07-06 20:06:02.786698723 +0000 UTC m=+15.913610827" Jul 06 20:06:02.804646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:02.804608 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mpdph" podStartSLOduration=3.5200000019999997 podStartE2EDuration="14.804595676s" totalImagesPullingTime="11.284595674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:02.804536966 +0000 UTC m=+15.931449071" watchObservedRunningTime="2026-07-06 20:06:02.804595676 +0000 UTC m=+15.931507781" Jul 06 20:06:03.212339 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:03.212267 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wvqlx" Jul 06 20:06:03.212815 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:03.212793 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wvqlx" Jul 06 20:06:03.478845 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:03.478751 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:03.478845 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:03.478765 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:03.479061 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:03.478893 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:03.479061 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:03.478913 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:06:03.479061 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:03.478996 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:06:03.479205 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:03.479068 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:06:04.698432 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:04.698405 2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 20:06:04.981615 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:04.981409 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs\") pod \"network-metrics-daemon-ztl9r\" (UID: \"8aa84d62-1d3e-4c96-8f8b-d78eb9650175\") " pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:04.981615 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:04.981557 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:06:04.981862 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:04.981627 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs podName:8aa84d62-1d3e-4c96-8f8b-d78eb9650175 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:20.981607783 +0000 UTC m=+34.108519879 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs") pod "network-metrics-daemon-ztl9r" (UID: "8aa84d62-1d3e-4c96-8f8b-d78eb9650175") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:06:05.082506 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:05.082471 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:05.082656 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:05.082594 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:06:05.082729 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:05.082657 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret podName:0b98a719-eb47-4991-93e9-271e9a4f0307 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:21.082639299 +0000 UTC m=+34.209551382 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret") pod "global-pull-secret-syncer-r6ptl" (UID: "0b98a719-eb47-4991-93e9-271e9a4f0307") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:06:05.183747 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:05.183714 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9dvl\" (UniqueName: \"kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl\") pod \"network-check-target-spktr\" (UID: \"0e1abbba-697e-4907-999a-848e5c5ea695\") " pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:05.183921 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:05.183905 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:06:05.184000 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:05.183925 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:06:05.184000 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:05.183935 2583 projected.go:196] Error preparing data for projected volume kube-api-access-h9dvl for pod openshift-network-diagnostics/network-check-target-spktr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:06:05.184000 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:05.183993 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl podName:0e1abbba-697e-4907-999a-848e5c5ea695 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:21.183976218 +0000 UTC m=+34.310888303 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-h9dvl" (UniqueName: "kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl") pod "network-check-target-spktr" (UID: "0e1abbba-697e-4907-999a-848e5c5ea695") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:06:05.479058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:05.478984 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:05.479228 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:05.478985 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:05.479228 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:05.479107 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:06:05.479228 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:05.479219 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:06:05.479367 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:05.479267 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:05.479997 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:05.479816 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:06:05.702534 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:05.702496 2583 generic.go:356] "Generic (PLEG): container finished" podID="b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4" containerID="85edde0ee68fde6ce11aed5e9b58694a6e308f8e0571f3c1df966c7a30618e7e" exitCode=0 Jul 06 20:06:05.703001 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:05.702570 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jp5d" event={"ID":"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4","Type":"ContainerDied","Data":"85edde0ee68fde6ce11aed5e9b58694a6e308f8e0571f3c1df966c7a30618e7e"} Jul 06 20:06:05.704910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:05.704890 2583 generic.go:356] "Generic (PLEG): container finished" podID="d927aedbafdf82c88fcd9938008cb116" containerID="2a5ac593305624a2724500eb074ca5fca134548d79eca78d20ee5a69a6f97e1c" exitCode=0 Jul 06 20:06:05.705021 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:05.704918 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" event={"ID":"d927aedbafdf82c88fcd9938008cb116","Type":"ContainerDied","Data":"2a5ac593305624a2724500eb074ca5fca134548d79eca78d20ee5a69a6f97e1c"} Jul 06 20:06:07.022824 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:07.022793 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wvqlx" Jul 06 20:06:07.023313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:07.022925 2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 20:06:07.023367 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:07.023353 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wvqlx" Jul 06 20:06:07.480097 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:07.480019 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:07.480097 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:07.480109 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:07.480326 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:07.480149 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:07.480326 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:07.480172 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:06:07.480326 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:07.480223 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:06:07.480326 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:07.480306 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:06:09.479076 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:09.479051 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:09.479076 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:09.479068 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:09.479585 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:09.479056 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:09.479585 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:09.479145 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:06:09.479585 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:09.479226 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:06:09.479585 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:09.479286 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:06:09.713825 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:09.713795 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-65r94" event={"ID":"f8f276d9-9aed-42a3-8e16-04d4a5e5eba5","Type":"ContainerStarted","Data":"fbc2518451c96f8e0daf757a51e6b3c3e9868618a29b18e7a9445c4b08c2028a"} Jul 06 20:06:10.232102 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.231932 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 20:06:10.342436 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.342241 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T20:06:10.232091272Z","UUID":"09cc4c81-31a6-416e-84ac-7d05db8c4055","Handler":null,"Name":"","Endpoint":""} Jul 06 20:06:10.343444 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.343428 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 20:06:10.343522 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.343460 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 20:06:10.719692 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.719615 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" event={"ID":"d927aedbafdf82c88fcd9938008cb116","Type":"ContainerStarted","Data":"6d6c4c0a2ca2bcb35f34263d7516a6a39353f4c6fdb4413a735baf5c623eef0d"} Jul 06 20:06:10.724559 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.724241 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" event={"ID":"e23596bc-6c8a-4070-b549-eec43455f210","Type":"ContainerStarted","Data":"407517875bd6e05ae4b73fae89918e41c8b56a02d607ebe6f4eb5fd3749a53d7"} Jul 06 20:06:10.724559 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.724318 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" event={"ID":"e23596bc-6c8a-4070-b549-eec43455f210","Type":"ContainerStarted","Data":"901fe74c8fb83bef22e39cd8f9808cf4824212dfe2b469f3e026f42f921cbeff"} Jul 06 20:06:10.724559 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.724367 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" event={"ID":"e23596bc-6c8a-4070-b549-eec43455f210","Type":"ContainerStarted","Data":"c8382dac2460302167f6a3124b6e9c51d2b1417678d9638c923668af3460a3b0"} Jul 06 20:06:10.724559 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.724413 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" event={"ID":"e23596bc-6c8a-4070-b549-eec43455f210","Type":"ContainerStarted","Data":"91e980457e96c9a2932370d62d807cd0af5dbe26c500307a3984a31817809a71"} Jul 06 20:06:10.724559 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.724460 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" event={"ID":"e23596bc-6c8a-4070-b549-eec43455f210","Type":"ContainerStarted","Data":"46012ed6bba5514f551279dbb69f7a601268d79d511ccef166f92e9130ca4c6c"} Jul 06 20:06:10.724559 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.724507 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" event={"ID":"e23596bc-6c8a-4070-b549-eec43455f210","Type":"ContainerStarted","Data":"08112c9d44328037acaf92ab948b07f3f7a3671bc7d16bf64b8553cffa69092c"} Jul 06 20:06:10.726656 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.726609 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" event={"ID":"770ab8d5-830a-4afe-9ee9-f453bf43a381","Type":"ContainerStarted","Data":"ec3a5262243315119e0e18d34a3d02b5681cedf724f3c0de71beb8225d774169"} Jul 06 20:06:10.728800 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.728710 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7t42j" event={"ID":"95ac569b-2c0f-4574-8bb7-38498017e7a5","Type":"ContainerStarted","Data":"5c18d12261373660bea2eb50373d7c0289ca7b351484e5035747736ca1abc320"} Jul 06 20:06:10.745263 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.745203 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-65r94" podStartSLOduration=12.310043037 podStartE2EDuration="23.745190082s" totalImagesPullingTime="11.435147045s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:09.747867465 +0000 UTC m=+22.874779570" watchObservedRunningTime="2026-07-06 20:06:10.745190082 +0000 UTC m=+23.872102186" Jul 06 20:06:10.745730 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.745695 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" podStartSLOduration=22.745687309 podStartE2EDuration="22.745687309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:05:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:10.745507547 +0000 UTC m=+23.872419678" watchObservedRunningTime="2026-07-06 20:06:10.745687309 +0000 UTC m=+23.872599412" Jul 06 20:06:10.769825 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:10.769776 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7t42j" podStartSLOduration=3.490203713 podStartE2EDuration="23.769762095s" totalImagesPullingTime="20.279558382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:10.769448911 +0000 UTC m=+23.896361016" watchObservedRunningTime="2026-07-06 20:06:10.769762095 +0000 UTC m=+23.896674218" Jul 06 20:06:11.479549 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:11.479529 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:11.479689 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:11.479529 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:11.479689 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:11.479623 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:06:11.479799 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:11.479529 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:11.479799 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:11.479700 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:06:11.479799 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:11.479763 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:06:11.733795 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:11.733701 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" event={"ID":"770ab8d5-830a-4afe-9ee9-f453bf43a381","Type":"ContainerStarted","Data":"5397da54efd4a995e70c4f3368f0e6c552b69572fc4464f7530365f030d18df3"} Jul 06 20:06:13.479052 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:13.479028 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:13.479491 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:13.479066 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:13.479491 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:13.479078 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:13.479491 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:13.479166 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:06:13.479491 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:13.479216 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:06:13.479491 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:13.479275 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:06:13.742004 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:13.741927 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" event={"ID":"e23596bc-6c8a-4070-b549-eec43455f210","Type":"ContainerStarted","Data":"8c0b1fa289fac92cd36c322423791b810183286adc4c8237c88ad75062a5e11d"} Jul 06 20:06:15.479739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:15.479571 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:15.480355 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:15.479571 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:15.480355 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:15.479812 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:06:15.480355 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:15.479633 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:15.480355 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:15.479893 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:06:15.480355 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:15.479965 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:06:15.749409 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:15.749336 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" event={"ID":"e23596bc-6c8a-4070-b549-eec43455f210","Type":"ContainerStarted","Data":"10cd3faca3a1b53cf90e33168de274d13b0bb8bc34412492cddd46bd764b4a1a"} Jul 06 20:06:15.749639 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:15.749624 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:06:15.751338 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:15.751297 2583 generic.go:356] "Generic (PLEG): container finished" podID="b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4" containerID="23c1106c3fd5b08d2e3504ad4fe85faaaacea60b4626da0313278b204c36e0b2" exitCode=0 Jul 06 20:06:15.751338 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:15.751325 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jp5d" event={"ID":"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4","Type":"ContainerDied","Data":"23c1106c3fd5b08d2e3504ad4fe85faaaacea60b4626da0313278b204c36e0b2"} Jul 06 20:06:15.764628 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:15.764609 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:06:15.789919 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:15.789861 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m6qc4" podStartSLOduration=7.87560052 podStartE2EDuration="27.789847403s" totalImagesPullingTime="19.914246883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:05:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:11.771525942 +0000 UTC m=+24.898438046" watchObservedRunningTime="2026-07-06 20:06:15.789847403 +0000 UTC m=+28.916759502" Jul 06 20:06:15.790393 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:15.790372 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" podStartSLOduration=7.480667135 podStartE2EDuration="27.790363747s" totalImagesPullingTime="20.309696612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:15.789924551 +0000 UTC m=+28.916836655" watchObservedRunningTime="2026-07-06 20:06:15.790363747 +0000 UTC m=+28.917275862" Jul 06 20:06:16.753211 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:16.753176 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:06:16.753711 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:16.753298 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:06:16.767293 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:16.767273 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:06:17.480739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:17.480709 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:17.480849 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:17.480786 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:17.480849 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:17.480826 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:17.480849 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:17.480822 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:06:17.480962 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:17.480923 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:06:17.481038 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:17.481019 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:06:17.537701 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:17.537680 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-spktr"] Jul 06 20:06:17.540262 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:17.540242 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ztl9r"] Jul 06 20:06:17.540936 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:17.540915 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r6ptl"] Jul 06 20:06:17.757229 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:17.757204 2583 generic.go:356] "Generic (PLEG): container finished" podID="b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4" containerID="aa45bff21efccf43e012e05a074fdd2c6fa74f99aa636a3eb4076d7cd74471a1" exitCode=0 Jul 06 20:06:17.757627 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:17.757289 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:17.757627 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:17.757312 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jp5d" event={"ID":"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4","Type":"ContainerDied","Data":"aa45bff21efccf43e012e05a074fdd2c6fa74f99aa636a3eb4076d7cd74471a1"} Jul 06 20:06:17.757627 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:17.757289 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:17.757627 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:17.757369 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:06:17.757627 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:17.757322 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:17.757627 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:17.757463 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:06:17.757969 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:17.757861 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:06:19.479645 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:19.479504 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:19.480067 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:19.479504 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:19.480067 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:19.479721 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:06:19.480067 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:19.479515 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:19.480067 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:19.479796 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:06:19.480067 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:19.479915 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:06:19.763777 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:19.763712 2583 generic.go:356] "Generic (PLEG): container finished" podID="b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4" containerID="92e5c21b2b262b7cc9e03e3436098c60616ccbd7541508259deeb156b89f430a" exitCode=0 Jul 06 20:06:19.763777 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:19.763745 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jp5d" event={"ID":"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4","Type":"ContainerDied","Data":"92e5c21b2b262b7cc9e03e3436098c60616ccbd7541508259deeb156b89f430a"} Jul 06 20:06:21.002145 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:21.002119 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs\") pod \"network-metrics-daemon-ztl9r\" (UID: \"8aa84d62-1d3e-4c96-8f8b-d78eb9650175\") " pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:21.002630 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:21.002228 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:06:21.002630 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:21.002275 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs podName:8aa84d62-1d3e-4c96-8f8b-d78eb9650175 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:53.002262494 +0000 UTC m=+66.129174579 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs") pod "network-metrics-daemon-ztl9r" (UID: "8aa84d62-1d3e-4c96-8f8b-d78eb9650175") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 20:06:21.102909 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:21.102869 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:21.103061 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:21.102970 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 20:06:21.103061 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:21.103040 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret podName:0b98a719-eb47-4991-93e9-271e9a4f0307 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:53.103019817 +0000 UTC m=+66.229931902 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret") pod "global-pull-secret-syncer-r6ptl" (UID: "0b98a719-eb47-4991-93e9-271e9a4f0307") : object "kube-system"/"original-pull-secret" not registered Jul 06 20:06:21.204051 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:21.204020 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9dvl\" (UniqueName: \"kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl\") pod \"network-check-target-spktr\" (UID: \"0e1abbba-697e-4907-999a-848e5c5ea695\") " pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:21.204178 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:21.204161 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 20:06:21.204246 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:21.204182 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 20:06:21.204246 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:21.204195 2583 projected.go:196] Error preparing data for projected volume kube-api-access-h9dvl for pod openshift-network-diagnostics/network-check-target-spktr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:06:21.204347 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:21.204253 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl podName:0e1abbba-697e-4907-999a-848e5c5ea695 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:53.204235431 +0000 UTC m=+66.331147516 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-h9dvl" (UniqueName: "kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl") pod "network-check-target-spktr" (UID: "0e1abbba-697e-4907-999a-848e5c5ea695") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 20:06:21.479218 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:21.479165 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:21.479516 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:21.479173 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:21.479516 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:21.479307 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:06:21.479516 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:21.479396 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:21.479516 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:21.479390 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:06:21.479516 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:21.479496 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:06:23.479547 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.479505 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:23.479547 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.479524 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:23.480051 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:23.479632 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6ptl" podUID="0b98a719-eb47-4991-93e9-271e9a4f0307" Jul 06 20:06:23.480051 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.479652 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:23.480051 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:23.479741 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztl9r" podUID="8aa84d62-1d3e-4c96-8f8b-d78eb9650175" Jul 06 20:06:23.480051 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:23.479816 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spktr" podUID="0e1abbba-697e-4907-999a-848e5c5ea695" Jul 06 20:06:23.685591 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.685522 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeReady" Jul 06 20:06:23.685759 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.685646 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 20:06:23.740123 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.740092 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-58d6568c64-ngz78"] Jul 06 20:06:23.765593 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.765560 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ppjpz"] Jul 06 20:06:23.765759 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.765691 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:23.769104 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.769083 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 20:06:23.769243 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.769109 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 20:06:23.769243 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.769089 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 20:06:23.769350 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.769252 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gmdh4\"" Jul 06 20:06:23.777473 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.777444 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 20:06:23.791637 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.791610 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl"] Jul 06 20:06:23.792917 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.792213 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ppjpz" Jul 06 20:06:23.795382 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.795242 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zbbrv\"" Jul 06 20:06:23.795382 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.795266 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 20:06:23.795563 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.795543 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:06:23.807624 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.807603 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg"] Jul 06 20:06:23.807762 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.807744 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" Jul 06 20:06:23.810614 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.810595 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 06 20:06:23.810614 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.810603 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vrqns\"" Jul 06 20:06:23.810932 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.810915 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 06 20:06:23.811074 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.811047 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:06:23.825725 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.825704 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-54f2g"] Jul 06 20:06:23.825844 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.825826 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:23.828818 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.828798 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 20:06:23.828983 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.828802 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 20:06:23.829147 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.829128 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 20:06:23.830118 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.830099 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 20:06:23.830188 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.830099 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cfzf5\"" Jul 06 20:06:23.849820 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.849802 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g"] Jul 06 20:06:23.849926 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.849915 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:23.852652 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.852635 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lnw9t\"" Jul 06 20:06:23.852898 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.852868 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 20:06:23.853261 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.853244 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 20:06:23.853384 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.853368 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 20:06:23.853778 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.853761 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 20:06:23.858893 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.857911 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 20:06:23.871969 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.871951 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-dddkw"] Jul 06 20:06:23.872068 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.872054 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" Jul 06 20:06:23.875574 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.875555 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 06 20:06:23.876017 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.876001 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 06 20:06:23.876089 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.876010 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 06 20:06:23.876291 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.876276 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:06:23.876374 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.876283 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-b486l\"" Jul 06 20:06:23.896153 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.896133 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b"] Jul 06 20:06:23.896242 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.896229 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:23.898936 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.898919 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 20:06:23.899156 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.899142 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 20:06:23.899246 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.899205 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:06:23.899246 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.899216 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 20:06:23.899372 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.899267 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-pqjnw\"" Jul 06 20:06:23.906065 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.906050 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 20:06:23.920267 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.920249 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5d5cf567d5-sqrwr"] Jul 06 20:06:23.920363 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.920350 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" Jul 06 20:06:23.923269 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.923254 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 06 20:06:23.923412 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.923387 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 06 20:06:23.923920 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.923902 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:06:23.924008 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.923913 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 06 20:06:23.924133 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924119 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-4sjx4\"" Jul 06 20:06:23.924195 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924163 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:23.924195 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924186 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/03cefcbc-3546-4aea-bcac-33036307b703-image-registry-private-configuration\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:23.924318 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924210 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjpt2\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-kube-api-access-tjpt2\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:23.924318 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924268 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/03cefcbc-3546-4aea-bcac-33036307b703-registry-certificates\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:23.924318 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924302 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6fzfl\" (UID: \"6edaf5b0-e004-446f-8119-1228997d15be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" Jul 06 20:06:23.924455 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924327 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/03cefcbc-3546-4aea-bcac-33036307b703-trusted-ca\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:23.924455 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924353 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/03cefcbc-3546-4aea-bcac-33036307b703-installation-pull-secrets\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:23.924455 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924381 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt8m4\" (UniqueName: \"kubernetes.io/projected/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-kube-api-access-dt8m4\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:23.924455 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924410 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:23.924662 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924472 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-bound-sa-token\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:23.924662 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924507 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slfbb\" (UniqueName: \"kubernetes.io/projected/f69be79e-9600-4313-8c0e-88348a100a7c-kube-api-access-slfbb\") pod \"volume-data-source-validator-5d456fdbcd-ppjpz\" (UID: \"f69be79e-9600-4313-8c0e-88348a100a7c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ppjpz" Jul 06 20:06:23.924662 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924626 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5njz\" (UniqueName: \"kubernetes.io/projected/6edaf5b0-e004-446f-8119-1228997d15be-kube-api-access-f5njz\") pod \"cluster-samples-operator-56fcf86bf-6fzfl\" (UID: \"6edaf5b0-e004-446f-8119-1228997d15be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" Jul 06 20:06:23.924662 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924654 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/03cefcbc-3546-4aea-bcac-33036307b703-ca-trust-extracted\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:23.924864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.924680 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:23.944663 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.944618 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58d6568c64-ngz78"] Jul 06 20:06:23.944663 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.944641 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ppjpz"] Jul 06 20:06:23.944663 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.944659 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nzdbq"] Jul 06 20:06:23.944833 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.944723 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:23.947881 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.947856 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2vlhw\"" Jul 06 20:06:23.948238 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.948221 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 20:06:23.948358 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.948272 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 20:06:23.948358 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.948308 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 20:06:23.948358 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.948272 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 20:06:23.948550 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.948536 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 20:06:23.948736 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.948722 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 20:06:23.958740 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.958726 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn"] Jul 06 20:06:23.958829 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.958817 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:23.961607 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.961589 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 20:06:23.961686 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.961598 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5vcsc\"" Jul 06 20:06:23.961865 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.961852 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 20:06:23.973037 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.973019 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7qq9j"] Jul 06 20:06:23.973112 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.973024 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" Jul 06 20:06:23.975713 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.975690 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 06 20:06:23.976093 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.976076 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 06 20:06:23.976173 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.976092 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-btfvm\"" Jul 06 20:06:23.995625 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.995518 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-kgfjd"] Jul 06 20:06:23.995764 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.995750 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7qq9j" Jul 06 20:06:23.998508 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.998491 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 20:06:23.998814 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.998801 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 20:06:23.998952 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.998932 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 20:06:23.999172 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:23.999157 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kwpnh\"" Jul 06 20:06:24.017469 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.017453 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b"] Jul 06 20:06:24.017568 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.017477 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-54f2g"] Jul 06 20:06:24.017568 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.017492 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl"] Jul 06 20:06:24.017568 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.017509 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg"] Jul 06 20:06:24.017568 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.017521 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g"] Jul 06 20:06:24.017568 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.017532 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nzdbq"] Jul 06 20:06:24.017568 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.017559 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-dddkw"] Jul 06 20:06:24.017774 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.017575 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn"] Jul 06 20:06:24.017774 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.017587 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7qq9j"] Jul 06 20:06:24.017774 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.017598 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-kgfjd"] Jul 06 20:06:24.017774 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.017612 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d5cf567d5-sqrwr"] Jul 06 20:06:24.017774 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.017654 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-kgfjd" Jul 06 20:06:24.020254 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.020240 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 20:06:24.020579 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.020564 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 20:06:24.020637 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.020626 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-t898x\"" Jul 06 20:06:24.025523 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.025507 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/03cefcbc-3546-4aea-bcac-33036307b703-registry-certificates\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.025619 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.025533 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c91495de-ffbb-4d96-a6a8-5511288f7cd0-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-nln8b\" (UID: \"c91495de-ffbb-4d96-a6a8-5511288f7cd0\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" Jul 06 20:06:24.025619 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.025552 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db968f0a-d281-4b2b-8eaf-285d77c876b2-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.025619 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.025569 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/03cefcbc-3546-4aea-bcac-33036307b703-installation-pull-secrets\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.025619 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.025603 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt8m4\" (UniqueName: \"kubernetes.io/projected/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-kube-api-access-dt8m4\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:24.025833 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.025669 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0962b7d0-5a52-4745-a3c3-65afa1f24ca1-trusted-ca\") pod \"console-operator-68b757fb67-dddkw\" (UID: \"0962b7d0-5a52-4745-a3c3-65afa1f24ca1\") " pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:24.025833 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.025701 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c91495de-ffbb-4d96-a6a8-5511288f7cd0-config\") pod \"service-ca-operator-5d46d9ffd4-nln8b\" (UID: \"c91495de-ffbb-4d96-a6a8-5511288f7cd0\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" Jul 06 20:06:24.025833 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.025722 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b359f2f9-4eb6-47f8-90cf-f096ba41e852-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hbc8g\" (UID: \"b359f2f9-4eb6-47f8-90cf-f096ba41e852\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" Jul 06 20:06:24.025833 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.025771 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:24.025833 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.025807 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0962b7d0-5a52-4745-a3c3-65afa1f24ca1-config\") pod \"console-operator-68b757fb67-dddkw\" (UID: \"0962b7d0-5a52-4745-a3c3-65afa1f24ca1\") " pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:24.026125 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.025858 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 20:06:24.026125 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.025902 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slfbb\" (UniqueName: \"kubernetes.io/projected/f69be79e-9600-4313-8c0e-88348a100a7c-kube-api-access-slfbb\") pod \"volume-data-source-validator-5d456fdbcd-ppjpz\" (UID: \"f69be79e-9600-4313-8c0e-88348a100a7c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ppjpz" Jul 06 20:06:24.026125 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.025923 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls podName:7f52f514-81ea-40b8-b6aa-9ab51d3f3462 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:24.525908326 +0000 UTC m=+37.652820408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-mvwbg" (UID: "7f52f514-81ea-40b8-b6aa-9ab51d3f3462") : secret "cluster-monitoring-operator-tls" not found Jul 06 20:06:24.026125 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026077 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db968f0a-d281-4b2b-8eaf-285d77c876b2-tmp\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.026125 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026096 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db968f0a-d281-4b2b-8eaf-285d77c876b2-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.026384 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026137 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6fzfl\" (UID: \"6edaf5b0-e004-446f-8119-1228997d15be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" Jul 06 20:06:24.026384 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026136 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/03cefcbc-3546-4aea-bcac-33036307b703-registry-certificates\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.026384 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.026218 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:06:24.026384 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026238 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db968f0a-d281-4b2b-8eaf-285d77c876b2-serving-cert\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.026384 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026255 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p5b4\" (UniqueName: \"kubernetes.io/projected/0962b7d0-5a52-4745-a3c3-65afa1f24ca1-kube-api-access-8p5b4\") pod \"console-operator-68b757fb67-dddkw\" (UID: \"0962b7d0-5a52-4745-a3c3-65afa1f24ca1\") " pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:24.026384 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.026283 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls podName:6edaf5b0-e004-446f-8119-1228997d15be nodeName:}" failed. No retries permitted until 2026-07-06 20:06:24.526267016 +0000 UTC m=+37.653179113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-6fzfl" (UID: "6edaf5b0-e004-446f-8119-1228997d15be") : secret "samples-operator-tls" not found Jul 06 20:06:24.026384 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026306 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkdhr\" (UniqueName: \"kubernetes.io/projected/db968f0a-d281-4b2b-8eaf-285d77c876b2-kube-api-access-gkdhr\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.026384 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026338 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccdwz\" (UniqueName: \"kubernetes.io/projected/c91495de-ffbb-4d96-a6a8-5511288f7cd0-kube-api-access-ccdwz\") pod \"service-ca-operator-5d46d9ffd4-nln8b\" (UID: \"c91495de-ffbb-4d96-a6a8-5511288f7cd0\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" Jul 06 20:06:24.026384 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026366 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sztv8\" (UniqueName: \"kubernetes.io/projected/b359f2f9-4eb6-47f8-90cf-f096ba41e852-kube-api-access-sztv8\") pod \"kube-storage-version-migrator-operator-846bf668f7-hbc8g\" (UID: \"b359f2f9-4eb6-47f8-90cf-f096ba41e852\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" Jul 06 20:06:24.026807 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026397 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-bound-sa-token\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.026807 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026537 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/03cefcbc-3546-4aea-bcac-33036307b703-trusted-ca\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.026807 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026567 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/db968f0a-d281-4b2b-8eaf-285d77c876b2-snapshots\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.026807 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026597 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.026807 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026619 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5njz\" (UniqueName: \"kubernetes.io/projected/6edaf5b0-e004-446f-8119-1228997d15be-kube-api-access-f5njz\") pod \"cluster-samples-operator-56fcf86bf-6fzfl\" (UID: \"6edaf5b0-e004-446f-8119-1228997d15be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" Jul 06 20:06:24.026807 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026637 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b359f2f9-4eb6-47f8-90cf-f096ba41e852-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hbc8g\" (UID: \"b359f2f9-4eb6-47f8-90cf-f096ba41e852\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" Jul 06 20:06:24.026807 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026655 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/03cefcbc-3546-4aea-bcac-33036307b703-ca-trust-extracted\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.026807 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026687 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjpt2\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-kube-api-access-tjpt2\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.026807 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026703 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0962b7d0-5a52-4745-a3c3-65afa1f24ca1-serving-cert\") pod \"console-operator-68b757fb67-dddkw\" (UID: \"0962b7d0-5a52-4745-a3c3-65afa1f24ca1\") " pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:24.026807 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.026705 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 20:06:24.026807 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.026720 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58d6568c64-ngz78: secret "image-registry-tls" not found Jul 06 20:06:24.026807 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026729 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:24.026807 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.026750 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/03cefcbc-3546-4aea-bcac-33036307b703-image-registry-private-configuration\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.026807 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.026760 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls podName:03cefcbc-3546-4aea-bcac-33036307b703 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:24.526745752 +0000 UTC m=+37.653657840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls") pod "image-registry-58d6568c64-ngz78" (UID: "03cefcbc-3546-4aea-bcac-33036307b703") : secret "image-registry-tls" not found Jul 06 20:06:24.027620 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.027148 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/03cefcbc-3546-4aea-bcac-33036307b703-ca-trust-extracted\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.027620 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.027347 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/03cefcbc-3546-4aea-bcac-33036307b703-trusted-ca\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.027620 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.027436 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:24.029520 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.029499 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/03cefcbc-3546-4aea-bcac-33036307b703-image-registry-private-configuration\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.029591 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.029572 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/03cefcbc-3546-4aea-bcac-33036307b703-installation-pull-secrets\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.041967 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.041944 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt8m4\" (UniqueName: \"kubernetes.io/projected/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-kube-api-access-dt8m4\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:24.045057 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.045025 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slfbb\" (UniqueName: \"kubernetes.io/projected/f69be79e-9600-4313-8c0e-88348a100a7c-kube-api-access-slfbb\") pod \"volume-data-source-validator-5d456fdbcd-ppjpz\" (UID: \"f69be79e-9600-4313-8c0e-88348a100a7c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ppjpz" Jul 06 20:06:24.045148 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.045129 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjpt2\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-kube-api-access-tjpt2\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.045235 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.045173 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-bound-sa-token\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.045553 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.045531 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5njz\" (UniqueName: \"kubernetes.io/projected/6edaf5b0-e004-446f-8119-1228997d15be-kube-api-access-f5njz\") pod \"cluster-samples-operator-56fcf86bf-6fzfl\" (UID: \"6edaf5b0-e004-446f-8119-1228997d15be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" Jul 06 20:06:24.102250 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.102229 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ppjpz" Jul 06 20:06:24.127155 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127140 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0962b7d0-5a52-4745-a3c3-65afa1f24ca1-trusted-ca\") pod \"console-operator-68b757fb67-dddkw\" (UID: \"0962b7d0-5a52-4745-a3c3-65afa1f24ca1\") " pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:24.127226 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127163 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c91495de-ffbb-4d96-a6a8-5511288f7cd0-config\") pod \"service-ca-operator-5d46d9ffd4-nln8b\" (UID: \"c91495de-ffbb-4d96-a6a8-5511288f7cd0\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" Jul 06 20:06:24.127226 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127181 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b359f2f9-4eb6-47f8-90cf-f096ba41e852-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hbc8g\" (UID: \"b359f2f9-4eb6-47f8-90cf-f096ba41e852\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" Jul 06 20:06:24.127226 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127205 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0962b7d0-5a52-4745-a3c3-65afa1f24ca1-config\") pod \"console-operator-68b757fb67-dddkw\" (UID: \"0962b7d0-5a52-4745-a3c3-65afa1f24ca1\") " pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:24.127419 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127397 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm2cc\" (UniqueName: \"kubernetes.io/projected/efe2b855-7351-46d1-8cc7-a7d96ff57e80-kube-api-access-lm2cc\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.127475 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127437 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-stats-auth\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.127475 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127463 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.127584 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127534 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r264\" (UniqueName: \"kubernetes.io/projected/ce43eb8c-4158-49b9-867a-900edcdd00c9-kube-api-access-6r264\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:24.127584 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127572 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce43eb8c-4158-49b9-867a-900edcdd00c9-tmp-dir\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:24.127679 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127601 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-8s7zn\" (UID: \"a39e36ae-32ec-4151-93b5-bb6fa2b01a31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" Jul 06 20:06:24.127679 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127629 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db968f0a-d281-4b2b-8eaf-285d77c876b2-tmp\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.127679 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127655 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db968f0a-d281-4b2b-8eaf-285d77c876b2-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.128232 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127706 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db968f0a-d281-4b2b-8eaf-285d77c876b2-serving-cert\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.128232 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127730 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p5b4\" (UniqueName: \"kubernetes.io/projected/0962b7d0-5a52-4745-a3c3-65afa1f24ca1-kube-api-access-8p5b4\") pod \"console-operator-68b757fb67-dddkw\" (UID: \"0962b7d0-5a52-4745-a3c3-65afa1f24ca1\") " pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:24.128232 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127741 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c91495de-ffbb-4d96-a6a8-5511288f7cd0-config\") pod \"service-ca-operator-5d46d9ffd4-nln8b\" (UID: \"c91495de-ffbb-4d96-a6a8-5511288f7cd0\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" Jul 06 20:06:24.128232 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127756 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkdhr\" (UniqueName: \"kubernetes.io/projected/db968f0a-d281-4b2b-8eaf-285d77c876b2-kube-api-access-gkdhr\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.128232 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.127853 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0962b7d0-5a52-4745-a3c3-65afa1f24ca1-config\") pod \"console-operator-68b757fb67-dddkw\" (UID: \"0962b7d0-5a52-4745-a3c3-65afa1f24ca1\") " pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:24.128232 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128008 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7snt\" (UniqueName: \"kubernetes.io/projected/59096fca-4e28-4504-aaa4-f99361c31d44-kube-api-access-r7snt\") pod \"ingress-canary-7qq9j\" (UID: \"59096fca-4e28-4504-aaa4-f99361c31d44\") " pod="openshift-ingress-canary/ingress-canary-7qq9j" Jul 06 20:06:24.128232 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128024 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db968f0a-d281-4b2b-8eaf-285d77c876b2-tmp\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.128232 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128031 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccdwz\" (UniqueName: \"kubernetes.io/projected/c91495de-ffbb-4d96-a6a8-5511288f7cd0-kube-api-access-ccdwz\") pod \"service-ca-operator-5d46d9ffd4-nln8b\" (UID: \"c91495de-ffbb-4d96-a6a8-5511288f7cd0\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" Jul 06 20:06:24.128232 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128065 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0962b7d0-5a52-4745-a3c3-65afa1f24ca1-trusted-ca\") pod \"console-operator-68b757fb67-dddkw\" (UID: \"0962b7d0-5a52-4745-a3c3-65afa1f24ca1\") " pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:24.128232 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128074 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-default-certificate\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.128232 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128110 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.128232 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128143 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sztv8\" (UniqueName: \"kubernetes.io/projected/b359f2f9-4eb6-47f8-90cf-f096ba41e852-kube-api-access-sztv8\") pod \"kube-storage-version-migrator-operator-846bf668f7-hbc8g\" (UID: \"b359f2f9-4eb6-47f8-90cf-f096ba41e852\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" Jul 06 20:06:24.128232 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128176 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/db968f0a-d281-4b2b-8eaf-285d77c876b2-snapshots\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.128232 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128221 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b359f2f9-4eb6-47f8-90cf-f096ba41e852-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hbc8g\" (UID: \"b359f2f9-4eb6-47f8-90cf-f096ba41e852\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" Jul 06 20:06:24.128847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128246 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:24.128847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128274 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db968f0a-d281-4b2b-8eaf-285d77c876b2-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.128847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128287 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0962b7d0-5a52-4745-a3c3-65afa1f24ca1-serving-cert\") pod \"console-operator-68b757fb67-dddkw\" (UID: \"0962b7d0-5a52-4745-a3c3-65afa1f24ca1\") " pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:24.128847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128343 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-8s7zn\" (UID: \"a39e36ae-32ec-4151-93b5-bb6fa2b01a31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" Jul 06 20:06:24.128847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128535 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c91495de-ffbb-4d96-a6a8-5511288f7cd0-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-nln8b\" (UID: \"c91495de-ffbb-4d96-a6a8-5511288f7cd0\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" Jul 06 20:06:24.128847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128560 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db968f0a-d281-4b2b-8eaf-285d77c876b2-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.128847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128565 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/db968f0a-d281-4b2b-8eaf-285d77c876b2-snapshots\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.128847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128584 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59096fca-4e28-4504-aaa4-f99361c31d44-cert\") pod \"ingress-canary-7qq9j\" (UID: \"59096fca-4e28-4504-aaa4-f99361c31d44\") " pod="openshift-ingress-canary/ingress-canary-7qq9j" Jul 06 20:06:24.128847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128612 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce43eb8c-4158-49b9-867a-900edcdd00c9-config-volume\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:24.128847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.128637 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ffmp\" (UniqueName: \"kubernetes.io/projected/b2ff9401-4fab-4438-a0ce-64564b26e241-kube-api-access-4ffmp\") pod \"network-check-source-989f8b8dc-kgfjd\" (UID: \"b2ff9401-4fab-4438-a0ce-64564b26e241\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-kgfjd" Jul 06 20:06:24.129368 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.129347 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db968f0a-d281-4b2b-8eaf-285d77c876b2-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.129758 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.129733 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b359f2f9-4eb6-47f8-90cf-f096ba41e852-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hbc8g\" (UID: \"b359f2f9-4eb6-47f8-90cf-f096ba41e852\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" Jul 06 20:06:24.130004 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.129982 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b359f2f9-4eb6-47f8-90cf-f096ba41e852-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hbc8g\" (UID: \"b359f2f9-4eb6-47f8-90cf-f096ba41e852\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" Jul 06 20:06:24.130262 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.130239 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db968f0a-d281-4b2b-8eaf-285d77c876b2-serving-cert\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.130656 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.130638 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0962b7d0-5a52-4745-a3c3-65afa1f24ca1-serving-cert\") pod \"console-operator-68b757fb67-dddkw\" (UID: \"0962b7d0-5a52-4745-a3c3-65afa1f24ca1\") " pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:24.130814 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.130800 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c91495de-ffbb-4d96-a6a8-5511288f7cd0-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-nln8b\" (UID: \"c91495de-ffbb-4d96-a6a8-5511288f7cd0\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" Jul 06 20:06:24.138487 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.138464 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkdhr\" (UniqueName: \"kubernetes.io/projected/db968f0a-d281-4b2b-8eaf-285d77c876b2-kube-api-access-gkdhr\") pod \"insights-operator-5c847dcfc4-54f2g\" (UID: \"db968f0a-d281-4b2b-8eaf-285d77c876b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.140989 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.140971 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p5b4\" (UniqueName: \"kubernetes.io/projected/0962b7d0-5a52-4745-a3c3-65afa1f24ca1-kube-api-access-8p5b4\") pod \"console-operator-68b757fb67-dddkw\" (UID: \"0962b7d0-5a52-4745-a3c3-65afa1f24ca1\") " pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:24.142088 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.142066 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sztv8\" (UniqueName: \"kubernetes.io/projected/b359f2f9-4eb6-47f8-90cf-f096ba41e852-kube-api-access-sztv8\") pod \"kube-storage-version-migrator-operator-846bf668f7-hbc8g\" (UID: \"b359f2f9-4eb6-47f8-90cf-f096ba41e852\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" Jul 06 20:06:24.142215 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.142200 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccdwz\" (UniqueName: \"kubernetes.io/projected/c91495de-ffbb-4d96-a6a8-5511288f7cd0-kube-api-access-ccdwz\") pod \"service-ca-operator-5d46d9ffd4-nln8b\" (UID: \"c91495de-ffbb-4d96-a6a8-5511288f7cd0\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" Jul 06 20:06:24.158261 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.158247 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" Jul 06 20:06:24.180385 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.180368 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" Jul 06 20:06:24.204365 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.204352 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:24.228720 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.228704 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" Jul 06 20:06:24.228886 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.228859 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7snt\" (UniqueName: \"kubernetes.io/projected/59096fca-4e28-4504-aaa4-f99361c31d44-kube-api-access-r7snt\") pod \"ingress-canary-7qq9j\" (UID: \"59096fca-4e28-4504-aaa4-f99361c31d44\") " pod="openshift-ingress-canary/ingress-canary-7qq9j" Jul 06 20:06:24.228944 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.228913 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-default-certificate\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.228980 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.228942 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.229026 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.228987 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:24.229064 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.229039 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-8s7zn\" (UID: \"a39e36ae-32ec-4151-93b5-bb6fa2b01a31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" Jul 06 20:06:24.229104 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.229069 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59096fca-4e28-4504-aaa4-f99361c31d44-cert\") pod \"ingress-canary-7qq9j\" (UID: \"59096fca-4e28-4504-aaa4-f99361c31d44\") " pod="openshift-ingress-canary/ingress-canary-7qq9j" Jul 06 20:06:24.229104 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.229087 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle podName:efe2b855-7351-46d1-8cc7-a7d96ff57e80 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:24.729069025 +0000 UTC m=+37.855981124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle") pod "router-default-5d5cf567d5-sqrwr" (UID: "efe2b855-7351-46d1-8cc7-a7d96ff57e80") : configmap references non-existent config key: service-ca.crt Jul 06 20:06:24.229202 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.229138 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce43eb8c-4158-49b9-867a-900edcdd00c9-config-volume\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:24.229202 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.229166 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:06:24.229202 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.229172 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ffmp\" (UniqueName: \"kubernetes.io/projected/b2ff9401-4fab-4438-a0ce-64564b26e241-kube-api-access-4ffmp\") pod \"network-check-source-989f8b8dc-kgfjd\" (UID: \"b2ff9401-4fab-4438-a0ce-64564b26e241\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-kgfjd" Jul 06 20:06:24.229342 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.229246 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls podName:ce43eb8c-4158-49b9-867a-900edcdd00c9 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:24.729209952 +0000 UTC m=+37.856122048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls") pod "dns-default-nzdbq" (UID: "ce43eb8c-4158-49b9-867a-900edcdd00c9") : secret "dns-default-metrics-tls" not found Jul 06 20:06:24.229342 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.229311 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm2cc\" (UniqueName: \"kubernetes.io/projected/efe2b855-7351-46d1-8cc7-a7d96ff57e80-kube-api-access-lm2cc\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.229342 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.229339 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-stats-auth\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.229491 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.229363 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.229491 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.229397 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r264\" (UniqueName: \"kubernetes.io/projected/ce43eb8c-4158-49b9-867a-900edcdd00c9-kube-api-access-6r264\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:24.229588 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.229576 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 20:06:24.229643 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.229632 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs podName:efe2b855-7351-46d1-8cc7-a7d96ff57e80 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:24.729616741 +0000 UTC m=+37.856528825 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs") pod "router-default-5d5cf567d5-sqrwr" (UID: "efe2b855-7351-46d1-8cc7-a7d96ff57e80") : secret "router-metrics-certs-default" not found Jul 06 20:06:24.229688 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.229665 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-8s7zn\" (UID: \"a39e36ae-32ec-4151-93b5-bb6fa2b01a31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" Jul 06 20:06:24.229737 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.229691 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce43eb8c-4158-49b9-867a-900edcdd00c9-tmp-dir\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:24.229737 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.229725 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-8s7zn\" (UID: \"a39e36ae-32ec-4151-93b5-bb6fa2b01a31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" Jul 06 20:06:24.229841 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.229827 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 20:06:24.229907 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.229899 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert podName:a39e36ae-32ec-4151-93b5-bb6fa2b01a31 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:24.729863017 +0000 UTC m=+37.856775114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-8s7zn" (UID: "a39e36ae-32ec-4151-93b5-bb6fa2b01a31") : secret "networking-console-plugin-cert" not found Jul 06 20:06:24.231240 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.231219 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59096fca-4e28-4504-aaa4-f99361c31d44-cert\") pod \"ingress-canary-7qq9j\" (UID: \"59096fca-4e28-4504-aaa4-f99361c31d44\") " pod="openshift-ingress-canary/ingress-canary-7qq9j" Jul 06 20:06:24.231315 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.231297 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-default-certificate\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.231444 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.231430 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-stats-auth\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.241752 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.241737 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce43eb8c-4158-49b9-867a-900edcdd00c9-tmp-dir\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:24.241862 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.241845 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce43eb8c-4158-49b9-867a-900edcdd00c9-config-volume\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:24.243493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.243477 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7snt\" (UniqueName: \"kubernetes.io/projected/59096fca-4e28-4504-aaa4-f99361c31d44-kube-api-access-r7snt\") pod \"ingress-canary-7qq9j\" (UID: \"59096fca-4e28-4504-aaa4-f99361c31d44\") " pod="openshift-ingress-canary/ingress-canary-7qq9j" Jul 06 20:06:24.246206 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.246182 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ffmp\" (UniqueName: \"kubernetes.io/projected/b2ff9401-4fab-4438-a0ce-64564b26e241-kube-api-access-4ffmp\") pod \"network-check-source-989f8b8dc-kgfjd\" (UID: \"b2ff9401-4fab-4438-a0ce-64564b26e241\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-kgfjd" Jul 06 20:06:24.246682 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.246665 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm2cc\" (UniqueName: \"kubernetes.io/projected/efe2b855-7351-46d1-8cc7-a7d96ff57e80-kube-api-access-lm2cc\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.246728 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.246669 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r264\" (UniqueName: \"kubernetes.io/projected/ce43eb8c-4158-49b9-867a-900edcdd00c9-kube-api-access-6r264\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:24.304019 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.304003 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7qq9j" Jul 06 20:06:24.325161 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.325142 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-kgfjd" Jul 06 20:06:24.533796 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.533761 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6fzfl\" (UID: \"6edaf5b0-e004-446f-8119-1228997d15be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" Jul 06 20:06:24.534273 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.533829 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:24.534273 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.534054 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 20:06:24.534273 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.534071 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58d6568c64-ngz78: secret "image-registry-tls" not found Jul 06 20:06:24.534273 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.534134 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls podName:03cefcbc-3546-4aea-bcac-33036307b703 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:25.534113331 +0000 UTC m=+38.661025435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls") pod "image-registry-58d6568c64-ngz78" (UID: "03cefcbc-3546-4aea-bcac-33036307b703") : secret "image-registry-tls" not found Jul 06 20:06:24.534492 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.534317 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:06:24.534492 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.534363 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls podName:6edaf5b0-e004-446f-8119-1228997d15be nodeName:}" failed. No retries permitted until 2026-07-06 20:06:25.534349352 +0000 UTC m=+38.661261454 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-6fzfl" (UID: "6edaf5b0-e004-446f-8119-1228997d15be") : secret "samples-operator-tls" not found Jul 06 20:06:24.534492 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.534424 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:24.534634 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.534572 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 20:06:24.534634 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.534611 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls podName:7f52f514-81ea-40b8-b6aa-9ab51d3f3462 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:25.534597271 +0000 UTC m=+38.661509370 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-mvwbg" (UID: "7f52f514-81ea-40b8-b6aa-9ab51d3f3462") : secret "cluster-monitoring-operator-tls" not found Jul 06 20:06:24.665622 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.665592 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-kgfjd"] Jul 06 20:06:24.669775 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.669755 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-54f2g"] Jul 06 20:06:24.670630 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.670611 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b"] Jul 06 20:06:24.674686 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.674066 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ppjpz"] Jul 06 20:06:24.675606 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.675550 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-dddkw"] Jul 06 20:06:24.676390 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.676374 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g"] Jul 06 20:06:24.688860 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.688839 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7qq9j"] Jul 06 20:06:24.721032 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:06:24.720817 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2ff9401_4fab_4438_a0ce_64564b26e241.slice/crio-d6c352cda764b673ba08643de513261d16384bbeae555bc8578b4cb032fb2652 WatchSource:0}: Error finding container d6c352cda764b673ba08643de513261d16384bbeae555bc8578b4cb032fb2652: Status 404 returned error can't find the container with id d6c352cda764b673ba08643de513261d16384bbeae555bc8578b4cb032fb2652 Jul 06 20:06:24.721459 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:06:24.721389 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf69be79e_9600_4313_8c0e_88348a100a7c.slice/crio-5cbd80b75f9adb7847fdfadd2a9e6f9ac007a1cfca0a9427e56ee2ce8f4bae55 WatchSource:0}: Error finding container 5cbd80b75f9adb7847fdfadd2a9e6f9ac007a1cfca0a9427e56ee2ce8f4bae55: Status 404 returned error can't find the container with id 5cbd80b75f9adb7847fdfadd2a9e6f9ac007a1cfca0a9427e56ee2ce8f4bae55 Jul 06 20:06:24.721696 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:06:24.721588 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb968f0a_d281_4b2b_8eaf_285d77c876b2.slice/crio-e4c1cfa3e45082422616897949af9317dfbf745be93c905edd2dc339b4c5bf97 WatchSource:0}: Error finding container e4c1cfa3e45082422616897949af9317dfbf745be93c905edd2dc339b4c5bf97: Status 404 returned error can't find the container with id e4c1cfa3e45082422616897949af9317dfbf745be93c905edd2dc339b4c5bf97 Jul 06 20:06:24.722386 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:06:24.722366 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59096fca_4e28_4504_aaa4_f99361c31d44.slice/crio-6b9b1e3c13fbb207d568e1e3b8decca0dd7a9f7c2d2be987985f370d8de7a2fc WatchSource:0}: Error finding container 6b9b1e3c13fbb207d568e1e3b8decca0dd7a9f7c2d2be987985f370d8de7a2fc: Status 404 returned error can't find the container with id 6b9b1e3c13fbb207d568e1e3b8decca0dd7a9f7c2d2be987985f370d8de7a2fc Jul 06 20:06:24.722914 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:06:24.722701 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0962b7d0_5a52_4745_a3c3_65afa1f24ca1.slice/crio-14974fba43c0d6ceded055a65a70a9846fd8175e5531f59e774921b50dd046ef WatchSource:0}: Error finding container 14974fba43c0d6ceded055a65a70a9846fd8175e5531f59e774921b50dd046ef: Status 404 returned error can't find the container with id 14974fba43c0d6ceded055a65a70a9846fd8175e5531f59e774921b50dd046ef Jul 06 20:06:24.725643 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:06:24.724117 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb359f2f9_4eb6_47f8_90cf_f096ba41e852.slice/crio-e8797deb5c3a25c357962067c82f35c7c135eaaaf3bf2bb8983b41e2bda03bf2 WatchSource:0}: Error finding container e8797deb5c3a25c357962067c82f35c7c135eaaaf3bf2bb8983b41e2bda03bf2: Status 404 returned error can't find the container with id e8797deb5c3a25c357962067c82f35c7c135eaaaf3bf2bb8983b41e2bda03bf2 Jul 06 20:06:24.736270 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.736246 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.736366 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.736301 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-8s7zn\" (UID: \"a39e36ae-32ec-4151-93b5-bb6fa2b01a31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" Jul 06 20:06:24.736366 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.736355 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:24.736465 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.736371 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 20:06:24.736465 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.736401 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:24.736465 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.736419 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs podName:efe2b855-7351-46d1-8cc7-a7d96ff57e80 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:25.736402447 +0000 UTC m=+38.863314549 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs") pod "router-default-5d5cf567d5-sqrwr" (UID: "efe2b855-7351-46d1-8cc7-a7d96ff57e80") : secret "router-metrics-certs-default" not found Jul 06 20:06:24.736465 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.736448 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 20:06:24.736762 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.736518 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:06:24.736762 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.736520 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert podName:a39e36ae-32ec-4151-93b5-bb6fa2b01a31 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:25.736503351 +0000 UTC m=+38.863415453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-8s7zn" (UID: "a39e36ae-32ec-4151-93b5-bb6fa2b01a31") : secret "networking-console-plugin-cert" not found Jul 06 20:06:24.736762 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.736560 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle podName:efe2b855-7351-46d1-8cc7-a7d96ff57e80 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:25.736551374 +0000 UTC m=+38.863463459 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle") pod "router-default-5d5cf567d5-sqrwr" (UID: "efe2b855-7351-46d1-8cc7-a7d96ff57e80") : configmap references non-existent config key: service-ca.crt Jul 06 20:06:24.736762 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:24.736569 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls podName:ce43eb8c-4158-49b9-867a-900edcdd00c9 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:25.736564184 +0000 UTC m=+38.863476266 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls") pod "dns-default-nzdbq" (UID: "ce43eb8c-4158-49b9-867a-900edcdd00c9") : secret "dns-default-metrics-tls" not found Jul 06 20:06:24.776776 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.776750 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" event={"ID":"c91495de-ffbb-4d96-a6a8-5511288f7cd0","Type":"ContainerStarted","Data":"915f335fbf8a60c90534486abf979ec309c0b4d2fe4ceedfdfd8e72b97f62a25"} Jul 06 20:06:24.777716 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.777691 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7qq9j" event={"ID":"59096fca-4e28-4504-aaa4-f99361c31d44","Type":"ContainerStarted","Data":"6b9b1e3c13fbb207d568e1e3b8decca0dd7a9f7c2d2be987985f370d8de7a2fc"} Jul 06 20:06:24.778666 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.778640 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" event={"ID":"db968f0a-d281-4b2b-8eaf-285d77c876b2","Type":"ContainerStarted","Data":"e4c1cfa3e45082422616897949af9317dfbf745be93c905edd2dc339b4c5bf97"} Jul 06 20:06:24.779741 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.779724 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-kgfjd" event={"ID":"b2ff9401-4fab-4438-a0ce-64564b26e241","Type":"ContainerStarted","Data":"d6c352cda764b673ba08643de513261d16384bbeae555bc8578b4cb032fb2652"} Jul 06 20:06:24.782165 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.782147 2583 generic.go:356] "Generic (PLEG): container finished" podID="b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4" containerID="d061774e236c387ce932a60834c8a987f449430e1a803681fd26b3825e04c12d" exitCode=0 Jul 06 20:06:24.782237 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.782201 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jp5d" event={"ID":"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4","Type":"ContainerDied","Data":"d061774e236c387ce932a60834c8a987f449430e1a803681fd26b3825e04c12d"} Jul 06 20:06:24.783180 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.783147 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" event={"ID":"b359f2f9-4eb6-47f8-90cf-f096ba41e852","Type":"ContainerStarted","Data":"e8797deb5c3a25c357962067c82f35c7c135eaaaf3bf2bb8983b41e2bda03bf2"} Jul 06 20:06:24.784152 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.784135 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ppjpz" event={"ID":"f69be79e-9600-4313-8c0e-88348a100a7c","Type":"ContainerStarted","Data":"5cbd80b75f9adb7847fdfadd2a9e6f9ac007a1cfca0a9427e56ee2ce8f4bae55"} Jul 06 20:06:24.785220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:24.785203 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-dddkw" event={"ID":"0962b7d0-5a52-4745-a3c3-65afa1f24ca1","Type":"ContainerStarted","Data":"14974fba43c0d6ceded055a65a70a9846fd8175e5531f59e774921b50dd046ef"} Jul 06 20:06:25.481865 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.481649 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:25.482308 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.482289 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:25.482777 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.482754 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:25.485487 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.485310 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 20:06:25.488531 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.488175 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 20:06:25.488531 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.488383 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mt2s9\"" Jul 06 20:06:25.489312 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.489164 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vntfk\"" Jul 06 20:06:25.543636 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.543610 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:25.544004 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.543711 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:25.544004 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.543821 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6fzfl\" (UID: \"6edaf5b0-e004-446f-8119-1228997d15be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" Jul 06 20:06:25.544004 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:25.543977 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:06:25.544151 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:25.544035 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls podName:6edaf5b0-e004-446f-8119-1228997d15be nodeName:}" failed. No retries permitted until 2026-07-06 20:06:27.54401724 +0000 UTC m=+40.670929337 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-6fzfl" (UID: "6edaf5b0-e004-446f-8119-1228997d15be") : secret "samples-operator-tls" not found Jul 06 20:06:25.544449 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:25.544429 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 20:06:25.544449 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:25.544451 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58d6568c64-ngz78: secret "image-registry-tls" not found Jul 06 20:06:25.544582 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:25.544499 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls podName:03cefcbc-3546-4aea-bcac-33036307b703 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:27.544483894 +0000 UTC m=+40.671395997 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls") pod "image-registry-58d6568c64-ngz78" (UID: "03cefcbc-3546-4aea-bcac-33036307b703") : secret "image-registry-tls" not found Jul 06 20:06:25.544582 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:25.544561 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 20:06:25.544715 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:25.544596 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls podName:7f52f514-81ea-40b8-b6aa-9ab51d3f3462 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:27.544585122 +0000 UTC m=+40.671497207 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-mvwbg" (UID: "7f52f514-81ea-40b8-b6aa-9ab51d3f3462") : secret "cluster-monitoring-operator-tls" not found Jul 06 20:06:25.747725 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.747650 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:25.747725 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.747717 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:25.747992 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.747825 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:25.747992 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.747890 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-8s7zn\" (UID: \"a39e36ae-32ec-4151-93b5-bb6fa2b01a31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" Jul 06 20:06:25.748087 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:25.748020 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 20:06:25.748087 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:25.748083 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert podName:a39e36ae-32ec-4151-93b5-bb6fa2b01a31 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:27.748064811 +0000 UTC m=+40.874976913 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-8s7zn" (UID: "a39e36ae-32ec-4151-93b5-bb6fa2b01a31") : secret "networking-console-plugin-cert" not found Jul 06 20:06:25.748563 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:25.748541 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle podName:efe2b855-7351-46d1-8cc7-a7d96ff57e80 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:27.7485218 +0000 UTC m=+40.875433888 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle") pod "router-default-5d5cf567d5-sqrwr" (UID: "efe2b855-7351-46d1-8cc7-a7d96ff57e80") : configmap references non-existent config key: service-ca.crt Jul 06 20:06:25.748669 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:25.748625 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:06:25.748669 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:25.748660 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls podName:ce43eb8c-4158-49b9-867a-900edcdd00c9 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:27.748649588 +0000 UTC m=+40.875561685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls") pod "dns-default-nzdbq" (UID: "ce43eb8c-4158-49b9-867a-900edcdd00c9") : secret "dns-default-metrics-tls" not found Jul 06 20:06:25.748774 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:25.748721 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 20:06:25.748774 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:25.748754 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs podName:efe2b855-7351-46d1-8cc7-a7d96ff57e80 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:27.748742502 +0000 UTC m=+40.875654591 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs") pod "router-default-5d5cf567d5-sqrwr" (UID: "efe2b855-7351-46d1-8cc7-a7d96ff57e80") : secret "router-metrics-certs-default" not found Jul 06 20:06:25.798908 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.798860 2583 generic.go:356] "Generic (PLEG): container finished" podID="b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4" containerID="7491ac2018605e741d643597f42db764686005bc7da0ea95d6f2ab87183f0fdc" exitCode=0 Jul 06 20:06:25.799082 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:25.798953 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jp5d" event={"ID":"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4","Type":"ContainerDied","Data":"7491ac2018605e741d643597f42db764686005bc7da0ea95d6f2ab87183f0fdc"} Jul 06 20:06:26.808403 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:26.807727 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8jp5d" event={"ID":"b9369d9b-5e6e-4ff9-9760-b79c93fb5cd4","Type":"ContainerStarted","Data":"b9007af5b6ff03e9e64052093796b6172d57ba40eabc82dc04aa9607edd85e6a"} Jul 06 20:06:26.847699 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:26.847274 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8jp5d" podStartSLOduration=12.201391022 podStartE2EDuration="39.847251864s" totalImagesPullingTime="27.645860842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:05:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:26.845834425 +0000 UTC m=+39.972746530" watchObservedRunningTime="2026-07-06 20:06:26.847251864 +0000 UTC m=+39.974163969" Jul 06 20:06:27.565941 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:27.565903 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6fzfl\" (UID: \"6edaf5b0-e004-446f-8119-1228997d15be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" Jul 06 20:06:27.566118 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:27.565972 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:27.566118 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:27.566043 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:27.566118 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:27.566066 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:06:27.566118 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:27.566106 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 20:06:27.566312 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:27.566123 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 20:06:27.566312 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:27.566125 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58d6568c64-ngz78: secret "image-registry-tls" not found Jul 06 20:06:27.566312 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:27.566143 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls podName:6edaf5b0-e004-446f-8119-1228997d15be nodeName:}" failed. No retries permitted until 2026-07-06 20:06:31.566121729 +0000 UTC m=+44.693033823 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-6fzfl" (UID: "6edaf5b0-e004-446f-8119-1228997d15be") : secret "samples-operator-tls" not found Jul 06 20:06:27.566312 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:27.566178 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls podName:03cefcbc-3546-4aea-bcac-33036307b703 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:31.566160664 +0000 UTC m=+44.693072761 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls") pod "image-registry-58d6568c64-ngz78" (UID: "03cefcbc-3546-4aea-bcac-33036307b703") : secret "image-registry-tls" not found Jul 06 20:06:27.566312 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:27.566202 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls podName:7f52f514-81ea-40b8-b6aa-9ab51d3f3462 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:31.566192864 +0000 UTC m=+44.693104946 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-mvwbg" (UID: "7f52f514-81ea-40b8-b6aa-9ab51d3f3462") : secret "cluster-monitoring-operator-tls" not found Jul 06 20:06:27.767639 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:27.767602 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:27.767808 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:27.767664 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-8s7zn\" (UID: \"a39e36ae-32ec-4151-93b5-bb6fa2b01a31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" Jul 06 20:06:27.767808 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:27.767722 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:27.767808 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:27.767747 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 20:06:27.767808 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:27.767773 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:27.768029 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:27.767820 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 20:06:27.768029 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:27.767828 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs podName:efe2b855-7351-46d1-8cc7-a7d96ff57e80 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:31.767809596 +0000 UTC m=+44.894721687 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs") pod "router-default-5d5cf567d5-sqrwr" (UID: "efe2b855-7351-46d1-8cc7-a7d96ff57e80") : secret "router-metrics-certs-default" not found Jul 06 20:06:27.768029 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:27.767921 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:06:27.768029 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:27.767933 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle podName:efe2b855-7351-46d1-8cc7-a7d96ff57e80 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:31.767919612 +0000 UTC m=+44.894831699 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle") pod "router-default-5d5cf567d5-sqrwr" (UID: "efe2b855-7351-46d1-8cc7-a7d96ff57e80") : configmap references non-existent config key: service-ca.crt Jul 06 20:06:27.768029 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:27.767968 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert podName:a39e36ae-32ec-4151-93b5-bb6fa2b01a31 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:31.76795577 +0000 UTC m=+44.894867856 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-8s7zn" (UID: "a39e36ae-32ec-4151-93b5-bb6fa2b01a31") : secret "networking-console-plugin-cert" not found Jul 06 20:06:27.768029 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:27.767987 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls podName:ce43eb8c-4158-49b9-867a-900edcdd00c9 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:31.767978266 +0000 UTC m=+44.894890355 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls") pod "dns-default-nzdbq" (UID: "ce43eb8c-4158-49b9-867a-900edcdd00c9") : secret "dns-default-metrics-tls" not found Jul 06 20:06:31.597910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:31.597855 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:31.598324 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:31.597961 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6fzfl\" (UID: \"6edaf5b0-e004-446f-8119-1228997d15be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" Jul 06 20:06:31.598324 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:31.598003 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:31.598324 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:31.598008 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 20:06:31.598324 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:31.598068 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls podName:7f52f514-81ea-40b8-b6aa-9ab51d3f3462 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:39.598051264 +0000 UTC m=+52.724963348 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-mvwbg" (UID: "7f52f514-81ea-40b8-b6aa-9ab51d3f3462") : secret "cluster-monitoring-operator-tls" not found Jul 06 20:06:31.598324 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:31.598108 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:06:31.598324 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:31.598160 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls podName:6edaf5b0-e004-446f-8119-1228997d15be nodeName:}" failed. No retries permitted until 2026-07-06 20:06:39.598147221 +0000 UTC m=+52.725059303 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-6fzfl" (UID: "6edaf5b0-e004-446f-8119-1228997d15be") : secret "samples-operator-tls" not found Jul 06 20:06:31.598324 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:31.598107 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 20:06:31.598324 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:31.598185 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58d6568c64-ngz78: secret "image-registry-tls" not found Jul 06 20:06:31.598324 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:31.598243 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls podName:03cefcbc-3546-4aea-bcac-33036307b703 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:39.598226288 +0000 UTC m=+52.725138387 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls") pod "image-registry-58d6568c64-ngz78" (UID: "03cefcbc-3546-4aea-bcac-33036307b703") : secret "image-registry-tls" not found Jul 06 20:06:31.799771 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:31.799748 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:31.799931 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:31.799849 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:31.799931 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:31.799888 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:06:31.799931 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:31.799909 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-8s7zn\" (UID: \"a39e36ae-32ec-4151-93b5-bb6fa2b01a31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" Jul 06 20:06:31.800073 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:31.799944 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls podName:ce43eb8c-4158-49b9-867a-900edcdd00c9 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:39.799929243 +0000 UTC m=+52.926841344 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls") pod "dns-default-nzdbq" (UID: "ce43eb8c-4158-49b9-867a-900edcdd00c9") : secret "dns-default-metrics-tls" not found Jul 06 20:06:31.800073 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:31.799972 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 20:06:31.800073 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:31.799982 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 20:06:31.800073 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:31.799992 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:31.800073 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:31.800030 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs podName:efe2b855-7351-46d1-8cc7-a7d96ff57e80 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:39.800017991 +0000 UTC m=+52.926930073 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs") pod "router-default-5d5cf567d5-sqrwr" (UID: "efe2b855-7351-46d1-8cc7-a7d96ff57e80") : secret "router-metrics-certs-default" not found Jul 06 20:06:31.800073 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:31.800045 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert podName:a39e36ae-32ec-4151-93b5-bb6fa2b01a31 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:39.800038662 +0000 UTC m=+52.926950745 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-8s7zn" (UID: "a39e36ae-32ec-4151-93b5-bb6fa2b01a31") : secret "networking-console-plugin-cert" not found Jul 06 20:06:31.800073 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:31.800072 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle podName:efe2b855-7351-46d1-8cc7-a7d96ff57e80 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:39.800064282 +0000 UTC m=+52.926976388 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle") pod "router-default-5d5cf567d5-sqrwr" (UID: "efe2b855-7351-46d1-8cc7-a7d96ff57e80") : configmap references non-existent config key: service-ca.crt Jul 06 20:06:32.826451 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:32.826178 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" event={"ID":"c91495de-ffbb-4d96-a6a8-5511288f7cd0","Type":"ContainerStarted","Data":"6f2908bcf08f94aa9edd7b2bacfcd896049687de32ac4e26c504aa1042c3e043"} Jul 06 20:06:32.827968 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:32.827942 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7qq9j" event={"ID":"59096fca-4e28-4504-aaa4-f99361c31d44","Type":"ContainerStarted","Data":"77adb4ab93742cc471604f3ec0fe17ca2315057ba63b7f714491be06b9b2a35b"} Jul 06 20:06:32.829898 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:32.829855 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" event={"ID":"db968f0a-d281-4b2b-8eaf-285d77c876b2","Type":"ContainerStarted","Data":"9d3980867bb15347bb610bd05160088b81027cca92aabfbc9e10e1006045f377"} Jul 06 20:06:32.831393 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:32.831368 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-kgfjd" event={"ID":"b2ff9401-4fab-4438-a0ce-64564b26e241","Type":"ContainerStarted","Data":"d268f8b81b421174bc84b2fea05921c869e42f8834feebd844a8f9b970c4b550"} Jul 06 20:06:32.833348 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:32.833324 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" event={"ID":"b359f2f9-4eb6-47f8-90cf-f096ba41e852","Type":"ContainerStarted","Data":"3f09e421716c6d7150cb388e653669f7ac348f25ffddba9300667a0041b14f7a"} Jul 06 20:06:32.834945 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:32.834890 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ppjpz" event={"ID":"f69be79e-9600-4313-8c0e-88348a100a7c","Type":"ContainerStarted","Data":"6452ab71d8f27a357c145b20f250d7c70e7485a1fd0c88f99736da61a068af49"} Jul 06 20:06:32.836541 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:32.836524 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/0.log" Jul 06 20:06:32.836622 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:32.836562 2583 generic.go:356] "Generic (PLEG): container finished" podID="0962b7d0-5a52-4745-a3c3-65afa1f24ca1" containerID="50139d80f5443451c95f2dbd2c235b6fc88dfc27de31fca65b2cd77464f4c908" exitCode=255 Jul 06 20:06:32.836622 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:32.836589 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-dddkw" event={"ID":"0962b7d0-5a52-4745-a3c3-65afa1f24ca1","Type":"ContainerDied","Data":"50139d80f5443451c95f2dbd2c235b6fc88dfc27de31fca65b2cd77464f4c908"} Jul 06 20:06:32.836811 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:32.836796 2583 scope.go:117] "RemoveContainer" containerID="50139d80f5443451c95f2dbd2c235b6fc88dfc27de31fca65b2cd77464f4c908" Jul 06 20:06:32.853344 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:32.852691 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" podStartSLOduration=30.516145318 podStartE2EDuration="37.852678357s" totalImagesPullingTime="7.336533039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:32.852644877 +0000 UTC m=+45.979556983" watchObservedRunningTime="2026-07-06 20:06:32.852678357 +0000 UTC m=+45.979590457" Jul 06 20:06:32.874731 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:32.874695 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" podStartSLOduration=30.541272214 podStartE2EDuration="37.874684344s" totalImagesPullingTime="7.33341213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:32.874236185 +0000 UTC m=+46.001148289" watchObservedRunningTime="2026-07-06 20:06:32.874684344 +0000 UTC m=+46.001596426" Jul 06 20:06:32.960416 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:32.960363 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7qq9j" podStartSLOduration=2.623279376 podStartE2EDuration="9.960344355s" totalImagesPullingTime="7.337064979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:06:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:32.924209763 +0000 UTC m=+46.051121870" watchObservedRunningTime="2026-07-06 20:06:32.960344355 +0000 UTC m=+46.087256460" Jul 06 20:06:32.962244 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:32.962110 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-kgfjd" podStartSLOduration=30.613958094 podStartE2EDuration="37.96209362s" totalImagesPullingTime="7.348135526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:32.95893645 +0000 UTC m=+46.085848554" watchObservedRunningTime="2026-07-06 20:06:32.96209362 +0000 UTC m=+46.089005728" Jul 06 20:06:33.005713 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.005673 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ppjpz" podStartSLOduration=30.699574276 podStartE2EDuration="38.005659189s" totalImagesPullingTime="7.306084913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:32.97966094 +0000 UTC m=+46.106573044" watchObservedRunningTime="2026-07-06 20:06:33.005659189 +0000 UTC m=+46.132571295" Jul 06 20:06:33.005809 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.005781 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" podStartSLOduration=30.668723788 podStartE2EDuration="38.005776234s" totalImagesPullingTime="7.337052446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:33.004454635 +0000 UTC m=+46.131366739" watchObservedRunningTime="2026-07-06 20:06:33.005776234 +0000 UTC m=+46.132688338" Jul 06 20:06:33.705226 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.705199 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kzg95"] Jul 06 20:06:33.721185 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.721163 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:33.724466 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.724448 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 20:06:33.724569 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.724452 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 20:06:33.724569 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.724507 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nkxmb\"" Jul 06 20:06:33.729503 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.729484 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kzg95"] Jul 06 20:06:33.819158 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.819130 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvz66\" (UniqueName: \"kubernetes.io/projected/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-api-access-rvz66\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:33.819302 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.819183 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:33.819302 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.819291 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-crio-socket\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:33.819404 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.819387 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-data-volume\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:33.819469 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.819454 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:33.841670 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.841644 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:06:33.842102 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.842087 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/0.log" Jul 06 20:06:33.842166 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.842122 2583 generic.go:356] "Generic (PLEG): container finished" podID="0962b7d0-5a52-4745-a3c3-65afa1f24ca1" containerID="aeb4c8082d7b6164fa725d570cd586853693eb5606cdf8505d04ce9cb898d528" exitCode=255 Jul 06 20:06:33.842217 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.842192 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-dddkw" event={"ID":"0962b7d0-5a52-4745-a3c3-65afa1f24ca1","Type":"ContainerDied","Data":"aeb4c8082d7b6164fa725d570cd586853693eb5606cdf8505d04ce9cb898d528"} Jul 06 20:06:33.853660 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.853639 2583 scope.go:117] "RemoveContainer" containerID="50139d80f5443451c95f2dbd2c235b6fc88dfc27de31fca65b2cd77464f4c908" Jul 06 20:06:33.853776 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.853740 2583 scope.go:117] "RemoveContainer" containerID="aeb4c8082d7b6164fa725d570cd586853693eb5606cdf8505d04ce9cb898d528" Jul 06 20:06:33.853954 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:33.853933 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-dddkw_openshift-console-operator(0962b7d0-5a52-4745-a3c3-65afa1f24ca1)\"" pod="openshift-console-operator/console-operator-68b757fb67-dddkw" podUID="0962b7d0-5a52-4745-a3c3-65afa1f24ca1" Jul 06 20:06:33.920332 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.920313 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-data-volume\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:33.921308 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.920557 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:33.921308 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.920657 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-data-volume\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:33.921308 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:33.920723 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 20:06:33.921308 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.920729 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvz66\" (UniqueName: \"kubernetes.io/projected/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-api-access-rvz66\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:33.921308 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:33.920772 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm podName:e95ceb16-8f18-43d5-8a9c-fce4e159bb0f nodeName:}" failed. No retries permitted until 2026-07-06 20:06:34.420753559 +0000 UTC m=+47.547665660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kzg95" (UID: "e95ceb16-8f18-43d5-8a9c-fce4e159bb0f") : configmap "kube-rbac-proxy" not found Jul 06 20:06:33.921308 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.920899 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:33.921308 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:33.921010 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 20:06:33.921308 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:33.921073 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls podName:e95ceb16-8f18-43d5-8a9c-fce4e159bb0f nodeName:}" failed. No retries permitted until 2026-07-06 20:06:34.421060184 +0000 UTC m=+47.547972266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kzg95" (UID: "e95ceb16-8f18-43d5-8a9c-fce4e159bb0f") : secret "insights-runtime-extractor-tls" not found Jul 06 20:06:33.921308 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.921108 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-crio-socket\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:33.921308 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.921303 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-crio-socket\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:33.935435 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:33.935410 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvz66\" (UniqueName: \"kubernetes.io/projected/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-api-access-rvz66\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:34.204822 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:34.204795 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:34.204985 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:34.204845 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:34.426319 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:34.426284 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:34.426493 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:34.426458 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 20:06:34.426572 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:34.426525 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:34.426636 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:34.426628 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls podName:e95ceb16-8f18-43d5-8a9c-fce4e159bb0f nodeName:}" failed. No retries permitted until 2026-07-06 20:06:35.426606692 +0000 UTC m=+48.553518779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kzg95" (UID: "e95ceb16-8f18-43d5-8a9c-fce4e159bb0f") : secret "insights-runtime-extractor-tls" not found Jul 06 20:06:34.426701 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:34.426632 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 20:06:34.426701 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:34.426688 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm podName:e95ceb16-8f18-43d5-8a9c-fce4e159bb0f nodeName:}" failed. No retries permitted until 2026-07-06 20:06:35.426675449 +0000 UTC m=+48.553587531 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kzg95" (UID: "e95ceb16-8f18-43d5-8a9c-fce4e159bb0f") : configmap "kube-rbac-proxy" not found Jul 06 20:06:34.847259 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:34.847234 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:06:34.847623 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:34.847557 2583 scope.go:117] "RemoveContainer" containerID="aeb4c8082d7b6164fa725d570cd586853693eb5606cdf8505d04ce9cb898d528" Jul 06 20:06:34.847711 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:34.847693 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-dddkw_openshift-console-operator(0962b7d0-5a52-4745-a3c3-65afa1f24ca1)\"" pod="openshift-console-operator/console-operator-68b757fb67-dddkw" podUID="0962b7d0-5a52-4745-a3c3-65afa1f24ca1" Jul 06 20:06:35.435862 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:35.435834 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:35.436017 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:35.435982 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 20:06:35.436058 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:35.436039 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 20:06:35.436058 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:35.436054 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls podName:e95ceb16-8f18-43d5-8a9c-fce4e159bb0f nodeName:}" failed. No retries permitted until 2026-07-06 20:06:37.436036008 +0000 UTC m=+50.562948108 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kzg95" (UID: "e95ceb16-8f18-43d5-8a9c-fce4e159bb0f") : secret "insights-runtime-extractor-tls" not found Jul 06 20:06:35.436131 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:35.436073 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm podName:e95ceb16-8f18-43d5-8a9c-fce4e159bb0f nodeName:}" failed. No retries permitted until 2026-07-06 20:06:37.436063681 +0000 UTC m=+50.562975777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kzg95" (UID: "e95ceb16-8f18-43d5-8a9c-fce4e159bb0f") : configmap "kube-rbac-proxy" not found Jul 06 20:06:35.436131 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:35.435982 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:35.852192 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:35.852170 2583 scope.go:117] "RemoveContainer" containerID="aeb4c8082d7b6164fa725d570cd586853693eb5606cdf8505d04ce9cb898d528" Jul 06 20:06:35.852544 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:35.852328 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-dddkw_openshift-console-operator(0962b7d0-5a52-4745-a3c3-65afa1f24ca1)\"" pod="openshift-console-operator/console-operator-68b757fb67-dddkw" podUID="0962b7d0-5a52-4745-a3c3-65afa1f24ca1" Jul 06 20:06:35.898827 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:35.898806 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mpdph_0fcc19e6-df9c-4444-981c-898f00976382/dns-node-resolver/0.log" Jul 06 20:06:36.899708 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:36.899677 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wm5jd_36bc4e9d-a829-4f2a-906e-754c4cef2a5d/node-ca/0.log" Jul 06 20:06:37.456692 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:37.456667 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:37.456850 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:37.456736 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:37.456850 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:37.456777 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 20:06:37.456850 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:37.456839 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm podName:e95ceb16-8f18-43d5-8a9c-fce4e159bb0f nodeName:}" failed. No retries permitted until 2026-07-06 20:06:41.456823467 +0000 UTC m=+54.583735549 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kzg95" (UID: "e95ceb16-8f18-43d5-8a9c-fce4e159bb0f") : configmap "kube-rbac-proxy" not found Jul 06 20:06:37.456850 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:37.456845 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 20:06:37.456999 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:37.456908 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls podName:e95ceb16-8f18-43d5-8a9c-fce4e159bb0f nodeName:}" failed. No retries permitted until 2026-07-06 20:06:41.456892494 +0000 UTC m=+54.583804580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kzg95" (UID: "e95ceb16-8f18-43d5-8a9c-fce4e159bb0f") : secret "insights-runtime-extractor-tls" not found Jul 06 20:06:37.500961 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:37.500938 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7qq9j_59096fca-4e28-4504-aaa4-f99361c31d44/serve-healthcheck-canary/0.log" Jul 06 20:06:39.676154 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:39.676117 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6fzfl\" (UID: \"6edaf5b0-e004-446f-8119-1228997d15be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" Jul 06 20:06:39.676531 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:39.676177 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:39.676531 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:39.676240 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:39.676531 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:39.676268 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 20:06:39.676531 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:39.676330 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 20:06:39.676531 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:39.676341 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 20:06:39.676531 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:39.676348 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58d6568c64-ngz78: secret "image-registry-tls" not found Jul 06 20:06:39.676531 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:39.676356 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls podName:6edaf5b0-e004-446f-8119-1228997d15be nodeName:}" failed. No retries permitted until 2026-07-06 20:06:55.676313682 +0000 UTC m=+68.803225764 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-6fzfl" (UID: "6edaf5b0-e004-446f-8119-1228997d15be") : secret "samples-operator-tls" not found Jul 06 20:06:39.676531 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:39.676379 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls podName:7f52f514-81ea-40b8-b6aa-9ab51d3f3462 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:55.676368084 +0000 UTC m=+68.803280168 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-mvwbg" (UID: "7f52f514-81ea-40b8-b6aa-9ab51d3f3462") : secret "cluster-monitoring-operator-tls" not found Jul 06 20:06:39.676531 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:39.676391 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls podName:03cefcbc-3546-4aea-bcac-33036307b703 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:55.676385193 +0000 UTC m=+68.803297275 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls") pod "image-registry-58d6568c64-ngz78" (UID: "03cefcbc-3546-4aea-bcac-33036307b703") : secret "image-registry-tls" not found Jul 06 20:06:39.877770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:39.877740 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:39.877935 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:39.877796 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:39.877935 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:39.877922 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle podName:efe2b855-7351-46d1-8cc7-a7d96ff57e80 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:55.877905057 +0000 UTC m=+69.004817159 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle") pod "router-default-5d5cf567d5-sqrwr" (UID: "efe2b855-7351-46d1-8cc7-a7d96ff57e80") : configmap references non-existent config key: service-ca.crt Jul 06 20:06:39.878018 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:39.877948 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:39.878018 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:39.877970 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 20:06:39.878018 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:39.877982 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-8s7zn\" (UID: \"a39e36ae-32ec-4151-93b5-bb6fa2b01a31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" Jul 06 20:06:39.878018 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:39.878014 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls podName:ce43eb8c-4158-49b9-867a-900edcdd00c9 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:55.878000245 +0000 UTC m=+69.004912326 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls") pod "dns-default-nzdbq" (UID: "ce43eb8c-4158-49b9-867a-900edcdd00c9") : secret "dns-default-metrics-tls" not found Jul 06 20:06:39.878150 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:39.878058 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 20:06:39.878150 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:39.878060 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 20:06:39.878150 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:39.878090 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert podName:a39e36ae-32ec-4151-93b5-bb6fa2b01a31 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:55.878082597 +0000 UTC m=+69.004994679 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-8s7zn" (UID: "a39e36ae-32ec-4151-93b5-bb6fa2b01a31") : secret "networking-console-plugin-cert" not found Jul 06 20:06:39.878150 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:39.878108 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs podName:efe2b855-7351-46d1-8cc7-a7d96ff57e80 nodeName:}" failed. No retries permitted until 2026-07-06 20:06:55.878097666 +0000 UTC m=+69.005009751 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs") pod "router-default-5d5cf567d5-sqrwr" (UID: "efe2b855-7351-46d1-8cc7-a7d96ff57e80") : secret "router-metrics-certs-default" not found Jul 06 20:06:41.491452 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:41.491422 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:41.491904 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:41.491492 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:41.491904 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:41.491600 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 20:06:41.491904 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:06:41.491663 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls podName:e95ceb16-8f18-43d5-8a9c-fce4e159bb0f nodeName:}" failed. No retries permitted until 2026-07-06 20:06:49.491647726 +0000 UTC m=+62.618559810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kzg95" (UID: "e95ceb16-8f18-43d5-8a9c-fce4e159bb0f") : secret "insights-runtime-extractor-tls" not found Jul 06 20:06:41.492036 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:41.491951 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:47.480094 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:47.480066 2583 scope.go:117] "RemoveContainer" containerID="aeb4c8082d7b6164fa725d570cd586853693eb5606cdf8505d04ce9cb898d528" Jul 06 20:06:47.888141 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:47.888069 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:06:47.888282 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:47.888147 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-dddkw" event={"ID":"0962b7d0-5a52-4745-a3c3-65afa1f24ca1","Type":"ContainerStarted","Data":"b35948d33b996ff54cc0fd760cc8143838b71e4d7019c77eeccc25853c631db4"} Jul 06 20:06:47.888426 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:47.888406 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:48.243405 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:48.243377 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-dddkw" Jul 06 20:06:48.287772 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:48.287720 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-dddkw" podStartSLOduration=45.95098196 podStartE2EDuration="53.28770313s" totalImagesPullingTime="7.33672117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:47.915200223 +0000 UTC m=+61.042112328" watchObservedRunningTime="2026-07-06 20:06:48.28770313 +0000 UTC m=+61.414615235" Jul 06 20:06:48.770176 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:48.770154 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4m62r" Jul 06 20:06:49.556155 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:49.556123 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:49.558369 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:49.558348 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95ceb16-8f18-43d5-8a9c-fce4e159bb0f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kzg95\" (UID: \"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f\") " pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:49.635806 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:49.635782 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nkxmb\"" Jul 06 20:06:49.641997 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:49.641980 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kzg95" Jul 06 20:06:49.775085 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:49.775005 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kzg95"] Jul 06 20:06:49.777154 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:06:49.777127 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode95ceb16_8f18_43d5_8a9c_fce4e159bb0f.slice/crio-a89d72cb81c31f51182933d5e80b4e7a4b07f47a8c07881cc570857f92e00867 WatchSource:0}: Error finding container a89d72cb81c31f51182933d5e80b4e7a4b07f47a8c07881cc570857f92e00867: Status 404 returned error can't find the container with id a89d72cb81c31f51182933d5e80b4e7a4b07f47a8c07881cc570857f92e00867 Jul 06 20:06:49.897620 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:49.897559 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kzg95" event={"ID":"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f","Type":"ContainerStarted","Data":"a89d72cb81c31f51182933d5e80b4e7a4b07f47a8c07881cc570857f92e00867"} Jul 06 20:06:53.092809 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.092775 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs\") pod \"network-metrics-daemon-ztl9r\" (UID: \"8aa84d62-1d3e-4c96-8f8b-d78eb9650175\") " pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:53.096415 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.096385 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 20:06:53.105954 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.105914 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8aa84d62-1d3e-4c96-8f8b-d78eb9650175-metrics-certs\") pod \"network-metrics-daemon-ztl9r\" (UID: \"8aa84d62-1d3e-4c96-8f8b-d78eb9650175\") " pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:53.117895 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.117859 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vntfk\"" Jul 06 20:06:53.125367 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.125256 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztl9r" Jul 06 20:06:53.194135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.194108 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:53.199739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.199720 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 20:06:53.206634 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.206615 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b98a719-eb47-4991-93e9-271e9a4f0307-original-pull-secret\") pod \"global-pull-secret-syncer-r6ptl\" (UID: \"0b98a719-eb47-4991-93e9-271e9a4f0307\") " pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:53.294651 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.294624 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9dvl\" (UniqueName: \"kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl\") pod \"network-check-target-spktr\" (UID: \"0e1abbba-697e-4907-999a-848e5c5ea695\") " pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:53.298337 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.298165 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9dvl\" (UniqueName: \"kubernetes.io/projected/0e1abbba-697e-4907-999a-848e5c5ea695-kube-api-access-h9dvl\") pod \"network-check-target-spktr\" (UID: \"0e1abbba-697e-4907-999a-848e5c5ea695\") " pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:53.407090 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.407029 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mt2s9\"" Jul 06 20:06:53.412626 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.412611 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:53.421862 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.421835 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6ptl" Jul 06 20:06:53.594985 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.594918 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ztl9r"] Jul 06 20:06:53.597303 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:06:53.597282 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aa84d62_1d3e_4c96_8f8b_d78eb9650175.slice/crio-0484d1341783138d042423dabfc364ed5d7d8d9f263dc0fbf6c7b40ce17925c5 WatchSource:0}: Error finding container 0484d1341783138d042423dabfc364ed5d7d8d9f263dc0fbf6c7b40ce17925c5: Status 404 returned error can't find the container with id 0484d1341783138d042423dabfc364ed5d7d8d9f263dc0fbf6c7b40ce17925c5 Jul 06 20:06:53.813301 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.813272 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-spktr"] Jul 06 20:06:53.814183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.814163 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r6ptl"] Jul 06 20:06:53.814559 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:06:53.814536 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1abbba_697e_4907_999a_848e5c5ea695.slice/crio-c7e57c42bea874e6941c9f5ee270e1b45bfa2d3ba1041ce4e040b78898f24292 WatchSource:0}: Error finding container c7e57c42bea874e6941c9f5ee270e1b45bfa2d3ba1041ce4e040b78898f24292: Status 404 returned error can't find the container with id c7e57c42bea874e6941c9f5ee270e1b45bfa2d3ba1041ce4e040b78898f24292 Jul 06 20:06:53.815067 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:06:53.815051 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b98a719_eb47_4991_93e9_271e9a4f0307.slice/crio-d87183612966d44ec9eec98c604ceb60b7d8c92677c1c5cb55fae14721caf060 WatchSource:0}: Error finding container d87183612966d44ec9eec98c604ceb60b7d8c92677c1c5cb55fae14721caf060: Status 404 returned error can't find the container with id d87183612966d44ec9eec98c604ceb60b7d8c92677c1c5cb55fae14721caf060 Jul 06 20:06:53.911434 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.911402 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r6ptl" event={"ID":"0b98a719-eb47-4991-93e9-271e9a4f0307","Type":"ContainerStarted","Data":"d87183612966d44ec9eec98c604ceb60b7d8c92677c1c5cb55fae14721caf060"} Jul 06 20:06:53.912676 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.912651 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ztl9r" event={"ID":"8aa84d62-1d3e-4c96-8f8b-d78eb9650175","Type":"ContainerStarted","Data":"0484d1341783138d042423dabfc364ed5d7d8d9f263dc0fbf6c7b40ce17925c5"} Jul 06 20:06:53.913725 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:53.913705 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-spktr" event={"ID":"0e1abbba-697e-4907-999a-848e5c5ea695","Type":"ContainerStarted","Data":"c7e57c42bea874e6941c9f5ee270e1b45bfa2d3ba1041ce4e040b78898f24292"} Jul 06 20:06:54.920357 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:54.920291 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kzg95" event={"ID":"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f","Type":"ContainerStarted","Data":"6028233c725e8d41f9fbbf785bd5bed77dd873038ec852338d67f7796c99a93e"} Jul 06 20:06:54.922475 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:54.922430 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-spktr" event={"ID":"0e1abbba-697e-4907-999a-848e5c5ea695","Type":"ContainerStarted","Data":"47046677d67c0b9eaf84bf069d401284d3ce30960482e5dbe1a819e9c346eaf2"} Jul 06 20:06:54.922723 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:54.922662 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:06:54.949036 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:54.948979 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-spktr" podStartSLOduration=67.948961639 podStartE2EDuration="1m7.948961639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:54.946489584 +0000 UTC m=+68.073401687" watchObservedRunningTime="2026-07-06 20:06:54.948961639 +0000 UTC m=+68.075873743" Jul 06 20:06:55.719396 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.719369 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:55.719573 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.719439 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6fzfl\" (UID: \"6edaf5b0-e004-446f-8119-1228997d15be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" Jul 06 20:06:55.719573 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.719486 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:55.722432 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.722407 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f52f514-81ea-40b8-b6aa-9ab51d3f3462-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-mvwbg\" (UID: \"7f52f514-81ea-40b8-b6aa-9ab51d3f3462\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:55.722551 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.722520 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls\") pod \"image-registry-58d6568c64-ngz78\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:55.722599 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.722563 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6edaf5b0-e004-446f-8119-1228997d15be-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-6fzfl\" (UID: \"6edaf5b0-e004-446f-8119-1228997d15be\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" Jul 06 20:06:55.880603 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.880573 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gmdh4\"" Jul 06 20:06:55.889118 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.889081 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:55.920698 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.920674 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:55.921100 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.920707 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-8s7zn\" (UID: \"a39e36ae-32ec-4151-93b5-bb6fa2b01a31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" Jul 06 20:06:55.921100 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.920764 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:55.921100 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.920793 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:55.921555 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.921340 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-vrqns\"" Jul 06 20:06:55.922255 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.922226 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efe2b855-7351-46d1-8cc7-a7d96ff57e80-service-ca-bundle\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:55.923741 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.923721 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efe2b855-7351-46d1-8cc7-a7d96ff57e80-metrics-certs\") pod \"router-default-5d5cf567d5-sqrwr\" (UID: \"efe2b855-7351-46d1-8cc7-a7d96ff57e80\") " pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:55.924235 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.924212 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a39e36ae-32ec-4151-93b5-bb6fa2b01a31-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-8s7zn\" (UID: \"a39e36ae-32ec-4151-93b5-bb6fa2b01a31\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" Jul 06 20:06:55.925791 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.925771 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce43eb8c-4158-49b9-867a-900edcdd00c9-metrics-tls\") pod \"dns-default-nzdbq\" (UID: \"ce43eb8c-4158-49b9-867a-900edcdd00c9\") " pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:55.928352 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.928332 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" Jul 06 20:06:55.928591 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.928562 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ztl9r" event={"ID":"8aa84d62-1d3e-4c96-8f8b-d78eb9650175","Type":"ContainerStarted","Data":"3f88c3ec9fd4d39f73c6fd2e88d62a13a18c42038fc50c4fa28721176356bede"} Jul 06 20:06:55.928700 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.928628 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ztl9r" event={"ID":"8aa84d62-1d3e-4c96-8f8b-d78eb9650175","Type":"ContainerStarted","Data":"7d295bf7840c6b4e613754118637a6835338025d6678decb066ecef1af5a729a"} Jul 06 20:06:55.930561 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.930498 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kzg95" event={"ID":"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f","Type":"ContainerStarted","Data":"5f8b8b86c6f078527e0ee4e5b1f322e694863eb38ddb1f9b3811918dddeb5e4f"} Jul 06 20:06:55.938668 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.938644 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cfzf5\"" Jul 06 20:06:55.946191 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.946157 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" Jul 06 20:06:55.964838 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:55.964758 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ztl9r" podStartSLOduration=67.528148216 podStartE2EDuration="1m8.964736227s" totalImagesPullingTime="1.436588011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:55.962701018 +0000 UTC m=+69.089613140" watchObservedRunningTime="2026-07-06 20:06:55.964736227 +0000 UTC m=+69.091648332" Jul 06 20:06:56.055146 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.054820 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2vlhw\"" Jul 06 20:06:56.063715 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.062970 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:56.069364 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.069343 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5vcsc\"" Jul 06 20:06:56.071886 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.071795 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58d6568c64-ngz78"] Jul 06 20:06:56.076914 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.076700 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nzdbq" Jul 06 20:06:56.077091 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:06:56.076931 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03cefcbc_3546_4aea_bcac_33036307b703.slice/crio-aedc500523ec9894fcc29bb70b267f291628d077dfaa8eb08a249c88b842e274 WatchSource:0}: Error finding container aedc500523ec9894fcc29bb70b267f291628d077dfaa8eb08a249c88b842e274: Status 404 returned error can't find the container with id aedc500523ec9894fcc29bb70b267f291628d077dfaa8eb08a249c88b842e274 Jul 06 20:06:56.085176 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.085145 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-btfvm\"" Jul 06 20:06:56.092465 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.092282 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" Jul 06 20:06:56.124304 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.124233 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl"] Jul 06 20:06:56.143506 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.141757 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg"] Jul 06 20:06:56.263902 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.263750 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d5cf567d5-sqrwr"] Jul 06 20:06:56.285941 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.285914 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nzdbq"] Jul 06 20:06:56.290993 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:06:56.290958 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce43eb8c_4158_49b9_867a_900edcdd00c9.slice/crio-b21289cd5af1642f0d65180378b3edd0f6823427a2a2d7892127ddf211831cc4 WatchSource:0}: Error finding container b21289cd5af1642f0d65180378b3edd0f6823427a2a2d7892127ddf211831cc4: Status 404 returned error can't find the container with id b21289cd5af1642f0d65180378b3edd0f6823427a2a2d7892127ddf211831cc4 Jul 06 20:06:56.333067 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.332945 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn"] Jul 06 20:06:56.938646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.938534 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" event={"ID":"efe2b855-7351-46d1-8cc7-a7d96ff57e80","Type":"ContainerStarted","Data":"aff0789b2b9a8880ec329e38a5007d7c1d8361a5f2fa5cd8333c617a2a18549f"} Jul 06 20:06:56.938646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.938595 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" event={"ID":"efe2b855-7351-46d1-8cc7-a7d96ff57e80","Type":"ContainerStarted","Data":"6d271846059e3857a1ff75e559254e53a653483f71c7e38493e5bdfd05b0345b"} Jul 06 20:06:56.940509 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.940399 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" event={"ID":"7f52f514-81ea-40b8-b6aa-9ab51d3f3462","Type":"ContainerStarted","Data":"9d30dbf706f138cb86f9c66349f9b03c1fe758cd34555d3c60a2614fcfa7a93c"} Jul 06 20:06:56.942513 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.942407 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" event={"ID":"6edaf5b0-e004-446f-8119-1228997d15be","Type":"ContainerStarted","Data":"6f3714919f6b360097ef831cc9ea655b05d68afef5a1588a63a7317d99746e09"} Jul 06 20:06:56.947675 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.947515 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58d6568c64-ngz78" event={"ID":"03cefcbc-3546-4aea-bcac-33036307b703","Type":"ContainerStarted","Data":"f1432994987f67990e81f54dc289c605cfbd417527b025f5205f15bbffeedd94"} Jul 06 20:06:56.947675 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.947558 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58d6568c64-ngz78" event={"ID":"03cefcbc-3546-4aea-bcac-33036307b703","Type":"ContainerStarted","Data":"aedc500523ec9894fcc29bb70b267f291628d077dfaa8eb08a249c88b842e274"} Jul 06 20:06:56.947675 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.947584 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:06:56.950441 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.950414 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" event={"ID":"a39e36ae-32ec-4151-93b5-bb6fa2b01a31","Type":"ContainerStarted","Data":"4419f9c3674aca233b333d56a6b0adc75941091f5e25ab6e43b8d54f9c40a4c3"} Jul 06 20:06:56.953592 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.953506 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nzdbq" event={"ID":"ce43eb8c-4158-49b9-867a-900edcdd00c9","Type":"ContainerStarted","Data":"b21289cd5af1642f0d65180378b3edd0f6823427a2a2d7892127ddf211831cc4"} Jul 06 20:06:56.969448 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:56.968920 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" podStartSLOduration=61.968902405 podStartE2EDuration="1m1.968902405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:56.967322898 +0000 UTC m=+70.094235032" watchObservedRunningTime="2026-07-06 20:06:56.968902405 +0000 UTC m=+70.095814507" Jul 06 20:06:57.001702 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:57.001001 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-58d6568c64-ngz78" podStartSLOduration=69.000982279 podStartE2EDuration="1m9.000982279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:06:57.000244705 +0000 UTC m=+70.127156834" watchObservedRunningTime="2026-07-06 20:06:57.000982279 +0000 UTC m=+70.127894402" Jul 06 20:06:57.063522 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:57.063495 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:57.066670 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:57.066503 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:57.958392 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:57.958323 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:06:57.960532 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:06:57.960342 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5d5cf567d5-sqrwr" Jul 06 20:07:02.983579 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:02.983546 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" event={"ID":"a39e36ae-32ec-4151-93b5-bb6fa2b01a31","Type":"ContainerStarted","Data":"5c50084083dc7a55be0abd65e921d0388fade47db37fa56b1cf890ed5dd03232"} Jul 06 20:07:02.985515 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:02.985490 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nzdbq" event={"ID":"ce43eb8c-4158-49b9-867a-900edcdd00c9","Type":"ContainerStarted","Data":"0615dffa87a03dd14252ec36ffee42d106ad706db46c45851eab449dc0ef4a64"} Jul 06 20:07:02.988415 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:02.988388 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kzg95" event={"ID":"e95ceb16-8f18-43d5-8a9c-fce4e159bb0f","Type":"ContainerStarted","Data":"9fcefef95b453689c2becffbc13b3ac458503f3e9f278fa808ae7286e4a84c22"} Jul 06 20:07:02.991548 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:02.991138 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" event={"ID":"7f52f514-81ea-40b8-b6aa-9ab51d3f3462","Type":"ContainerStarted","Data":"4ec8ece42ece7cd1b69199a7e599be8cf086e46fdcf1080abccc31a424f63a54"} Jul 06 20:07:02.994108 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:02.994079 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" event={"ID":"6edaf5b0-e004-446f-8119-1228997d15be","Type":"ContainerStarted","Data":"cc3c9f52c344ca8e9ed78a735af72787e54dc6f35171d0a34cf8ae1b9faf6c0b"} Jul 06 20:07:02.994208 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:02.994121 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" event={"ID":"6edaf5b0-e004-446f-8119-1228997d15be","Type":"ContainerStarted","Data":"489cf58f74a18ec82cb425c0dad69461c5fa0aa3e65a01f9e45dbf4bcc80b63e"} Jul 06 20:07:02.996110 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:02.996067 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r6ptl" event={"ID":"0b98a719-eb47-4991-93e9-271e9a4f0307","Type":"ContainerStarted","Data":"687366fd70b7e2c3520d67076931c95ada5fb237e5671a452473e621c941b49b"} Jul 06 20:07:03.008412 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:03.008339 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-8s7zn" podStartSLOduration=46.745517439 podStartE2EDuration="53.008323587s" totalImagesPullingTime="6.262806148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:07:03.007804263 +0000 UTC m=+76.134716368" watchObservedRunningTime="2026-07-06 20:07:03.008323587 +0000 UTC m=+76.135235693" Jul 06 20:07:03.032815 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:03.032762 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-mvwbg" podStartSLOduration=61.587276856 podStartE2EDuration="1m8.032744081s" totalImagesPullingTime="6.445467225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:07:03.032056938 +0000 UTC m=+76.158969045" watchObservedRunningTime="2026-07-06 20:07:03.032744081 +0000 UTC m=+76.159656185" Jul 06 20:07:03.064190 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:03.064136 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-6fzfl" podStartSLOduration=61.678741816 podStartE2EDuration="1m8.064114895s" totalImagesPullingTime="6.385373079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:07:03.062896533 +0000 UTC m=+76.189808638" watchObservedRunningTime="2026-07-06 20:07:03.064114895 +0000 UTC m=+76.191027000" Jul 06 20:07:03.090451 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:03.090403 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kzg95" podStartSLOduration=17.46629882 podStartE2EDuration="30.090389402s" totalImagesPullingTime="12.624090582s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:06:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:07:03.089669395 +0000 UTC m=+76.216581500" watchObservedRunningTime="2026-07-06 20:07:03.090389402 +0000 UTC m=+76.217301761" Jul 06 20:07:03.126065 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:03.125982 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-r6ptl" podStartSLOduration=66.352349586 podStartE2EDuration="1m15.125963456s" totalImagesPullingTime="8.77361387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:07:03.124366468 +0000 UTC m=+76.251278587" watchObservedRunningTime="2026-07-06 20:07:03.125963456 +0000 UTC m=+76.252875561" Jul 06 20:07:04.002071 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:04.002022 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nzdbq" event={"ID":"ce43eb8c-4158-49b9-867a-900edcdd00c9","Type":"ContainerStarted","Data":"a3447bcee392771b5ccbff63bdf55a3585effed01bd4d3425499001483720524"} Jul 06 20:07:04.026524 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:04.026466 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nzdbq" podStartSLOduration=34.714909635 podStartE2EDuration="41.026448534s" totalImagesPullingTime="6.311538899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:06:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:07:04.025933016 +0000 UTC m=+77.152845122" watchObservedRunningTime="2026-07-06 20:07:04.026448534 +0000 UTC m=+77.153360643" Jul 06 20:07:05.006634 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:05.006603 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nzdbq" Jul 06 20:07:06.776179 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.776150 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68497d5b9b-d298g"] Jul 06 20:07:06.778722 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.778695 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:06.783040 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.783021 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 20:07:06.783139 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.783113 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5gqqx\"" Jul 06 20:07:06.783324 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.783301 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 20:07:06.783423 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.783359 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 20:07:06.783485 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.783431 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 20:07:06.784201 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.784183 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 20:07:06.784527 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.784397 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 20:07:06.784689 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.784501 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 20:07:06.787485 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.787467 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 20:07:06.790401 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.790384 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68497d5b9b-d298g"] Jul 06 20:07:06.910060 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.910034 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-trusted-ca-bundle\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:06.910234 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.910065 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-service-ca\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:06.910234 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.910113 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c37deaf8-205f-428b-a120-0afffdabdbcd-console-oauth-config\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:06.910234 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.910131 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6m5g\" (UniqueName: \"kubernetes.io/projected/c37deaf8-205f-428b-a120-0afffdabdbcd-kube-api-access-r6m5g\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:06.910234 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.910149 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c37deaf8-205f-428b-a120-0afffdabdbcd-console-serving-cert\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:06.910234 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.910168 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-console-config\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:06.910426 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:06.910236 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-oauth-serving-cert\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.011363 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.011341 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c37deaf8-205f-428b-a120-0afffdabdbcd-console-oauth-config\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.011507 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.011369 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6m5g\" (UniqueName: \"kubernetes.io/projected/c37deaf8-205f-428b-a120-0afffdabdbcd-kube-api-access-r6m5g\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.011507 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.011388 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c37deaf8-205f-428b-a120-0afffdabdbcd-console-serving-cert\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.011507 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.011415 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-console-config\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.011507 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.011440 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-oauth-serving-cert\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.011719 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.011539 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-trusted-ca-bundle\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.011719 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.011579 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-service-ca\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.012272 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.012247 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-console-config\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.012397 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.012317 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-service-ca\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.012397 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.012317 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-oauth-serving-cert\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.012799 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.012776 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-trusted-ca-bundle\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.018535 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.014375 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c37deaf8-205f-428b-a120-0afffdabdbcd-console-serving-cert\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.018535 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.014390 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c37deaf8-205f-428b-a120-0afffdabdbcd-console-oauth-config\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.024002 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.023980 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6m5g\" (UniqueName: \"kubernetes.io/projected/c37deaf8-205f-428b-a120-0afffdabdbcd-kube-api-access-r6m5g\") pod \"console-68497d5b9b-d298g\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.088183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.088128 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:07.214230 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:07.214205 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68497d5b9b-d298g"] Jul 06 20:07:07.216651 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:07:07.216623 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc37deaf8_205f_428b_a120_0afffdabdbcd.slice/crio-85bcc3502b8b1955cda9c8e95467213562f361a71a95a4a5605da36f37b3cd46 WatchSource:0}: Error finding container 85bcc3502b8b1955cda9c8e95467213562f361a71a95a4a5605da36f37b3cd46: Status 404 returned error can't find the container with id 85bcc3502b8b1955cda9c8e95467213562f361a71a95a4a5605da36f37b3cd46 Jul 06 20:07:08.022048 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:08.022004 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68497d5b9b-d298g" event={"ID":"c37deaf8-205f-428b-a120-0afffdabdbcd","Type":"ContainerStarted","Data":"85bcc3502b8b1955cda9c8e95467213562f361a71a95a4a5605da36f37b3cd46"} Jul 06 20:07:11.035784 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:11.035753 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68497d5b9b-d298g" event={"ID":"c37deaf8-205f-428b-a120-0afffdabdbcd","Type":"ContainerStarted","Data":"e9631f7f7124f19190c790b49216f5af8ea52360cbc1d7666b727ad8f6b3f884"} Jul 06 20:07:11.069350 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:11.069301 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68497d5b9b-d298g" podStartSLOduration=2.003886944 podStartE2EDuration="5.069285631s" totalImagesPullingTime="3.065398687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:07:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:07:11.069069148 +0000 UTC m=+84.195981252" watchObservedRunningTime="2026-07-06 20:07:11.069285631 +0000 UTC m=+84.196197731" Jul 06 20:07:12.012463 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:12.012434 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nzdbq" Jul 06 20:07:17.088720 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:17.088688 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:17.089196 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:17.088886 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:17.093626 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:17.093606 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:17.961917 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:17.961889 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:07:18.063784 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:18.063758 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:07:18.489737 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:18.489703 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-58d6568c64-ngz78"] Jul 06 20:07:22.023356 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.023324 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2"] Jul 06 20:07:22.027954 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.027933 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" Jul 06 20:07:22.033703 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.033660 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 20:07:22.033826 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.033804 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-2g9b8\"" Jul 06 20:07:22.034707 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.034569 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 20:07:22.034707 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.034575 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 20:07:22.057847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.057820 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nsnl5"] Jul 06 20:07:22.062729 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.062708 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2"] Jul 06 20:07:22.062829 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.062813 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.065671 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.065653 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lbch4\"" Jul 06 20:07:22.066243 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.066226 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 20:07:22.066327 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.066257 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 20:07:22.067003 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.066987 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 20:07:22.081506 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.081485 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl"] Jul 06 20:07:22.086834 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.086819 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.089377 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.089359 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 20:07:22.089548 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.089534 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 20:07:22.090574 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.090559 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-gkc75\"" Jul 06 20:07:22.090699 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.090683 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 20:07:22.100575 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.100555 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl"] Jul 06 20:07:22.116653 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.116631 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/41eb1a6e-35a2-4a3b-8459-06ad50a80756-root\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.116748 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.116675 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-tls\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.116748 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.116708 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.116748 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.116737 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-accelerators-collector-config\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.116932 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.116764 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19582221-42b6-43fe-86ac-c5f1f2aa9e77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.116932 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.116795 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/19582221-42b6-43fe-86ac-c5f1f2aa9e77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.116932 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.116823 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41eb1a6e-35a2-4a3b-8459-06ad50a80756-metrics-client-ca\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.116932 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.116849 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19582221-42b6-43fe-86ac-c5f1f2aa9e77-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.116932 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.116892 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41eb1a6e-35a2-4a3b-8459-06ad50a80756-sys\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.117187 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.116953 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caaba1f6-5de3-45d3-bd70-c71692fd5877-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-dmrp2\" (UID: \"caaba1f6-5de3-45d3-bd70-c71692fd5877\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" Jul 06 20:07:22.117187 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.116988 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/19582221-42b6-43fe-86ac-c5f1f2aa9e77-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.117187 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.117036 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-textfile\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.117187 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.117066 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caaba1f6-5de3-45d3-bd70-c71692fd5877-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-dmrp2\" (UID: \"caaba1f6-5de3-45d3-bd70-c71692fd5877\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" Jul 06 20:07:22.117187 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.117095 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/caaba1f6-5de3-45d3-bd70-c71692fd5877-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-dmrp2\" (UID: \"caaba1f6-5de3-45d3-bd70-c71692fd5877\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" Jul 06 20:07:22.117187 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.117123 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr6p4\" (UniqueName: \"kubernetes.io/projected/41eb1a6e-35a2-4a3b-8459-06ad50a80756-kube-api-access-fr6p4\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.117187 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.117157 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-wtmp\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.117398 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.117206 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkfnp\" (UniqueName: \"kubernetes.io/projected/19582221-42b6-43fe-86ac-c5f1f2aa9e77-kube-api-access-wkfnp\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.117398 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.117233 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psdwf\" (UniqueName: \"kubernetes.io/projected/caaba1f6-5de3-45d3-bd70-c71692fd5877-kube-api-access-psdwf\") pod \"openshift-state-metrics-54cf96dfcf-dmrp2\" (UID: \"caaba1f6-5de3-45d3-bd70-c71692fd5877\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" Jul 06 20:07:22.117398 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.117255 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/19582221-42b6-43fe-86ac-c5f1f2aa9e77-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.217654 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.217631 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-accelerators-collector-config\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.217772 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.217665 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19582221-42b6-43fe-86ac-c5f1f2aa9e77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.217772 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.217684 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/19582221-42b6-43fe-86ac-c5f1f2aa9e77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.217772 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.217701 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41eb1a6e-35a2-4a3b-8459-06ad50a80756-metrics-client-ca\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.217772 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.217720 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19582221-42b6-43fe-86ac-c5f1f2aa9e77-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.217772 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.217733 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41eb1a6e-35a2-4a3b-8459-06ad50a80756-sys\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.218060 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.217796 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41eb1a6e-35a2-4a3b-8459-06ad50a80756-sys\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.218060 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.217831 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caaba1f6-5de3-45d3-bd70-c71692fd5877-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-dmrp2\" (UID: \"caaba1f6-5de3-45d3-bd70-c71692fd5877\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" Jul 06 20:07:22.218060 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.217856 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/19582221-42b6-43fe-86ac-c5f1f2aa9e77-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.218060 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.217905 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-textfile\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.218060 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.217937 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caaba1f6-5de3-45d3-bd70-c71692fd5877-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-dmrp2\" (UID: \"caaba1f6-5de3-45d3-bd70-c71692fd5877\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" Jul 06 20:07:22.218060 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.217964 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/caaba1f6-5de3-45d3-bd70-c71692fd5877-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-dmrp2\" (UID: \"caaba1f6-5de3-45d3-bd70-c71692fd5877\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" Jul 06 20:07:22.218060 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.217989 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr6p4\" (UniqueName: \"kubernetes.io/projected/41eb1a6e-35a2-4a3b-8459-06ad50a80756-kube-api-access-fr6p4\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.218060 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.218026 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-wtmp\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.218515 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.218065 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkfnp\" (UniqueName: \"kubernetes.io/projected/19582221-42b6-43fe-86ac-c5f1f2aa9e77-kube-api-access-wkfnp\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.218515 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.218105 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psdwf\" (UniqueName: \"kubernetes.io/projected/caaba1f6-5de3-45d3-bd70-c71692fd5877-kube-api-access-psdwf\") pod \"openshift-state-metrics-54cf96dfcf-dmrp2\" (UID: \"caaba1f6-5de3-45d3-bd70-c71692fd5877\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" Jul 06 20:07:22.218515 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.218130 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/19582221-42b6-43fe-86ac-c5f1f2aa9e77-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.218515 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.218160 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/41eb1a6e-35a2-4a3b-8459-06ad50a80756-root\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.218515 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.218198 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-tls\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.218515 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.218231 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.218515 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.218344 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-accelerators-collector-config\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.218515 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.218344 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41eb1a6e-35a2-4a3b-8459-06ad50a80756-metrics-client-ca\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.218936 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.218561 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/19582221-42b6-43fe-86ac-c5f1f2aa9e77-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.218936 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.218590 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-textfile\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.218936 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.218593 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19582221-42b6-43fe-86ac-c5f1f2aa9e77-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.218936 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.218636 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/41eb1a6e-35a2-4a3b-8459-06ad50a80756-root\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.218936 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.218817 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-wtmp\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.219132 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.219044 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/19582221-42b6-43fe-86ac-c5f1f2aa9e77-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.219360 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.219327 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caaba1f6-5de3-45d3-bd70-c71692fd5877-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-dmrp2\" (UID: \"caaba1f6-5de3-45d3-bd70-c71692fd5877\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" Jul 06 20:07:22.220739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.220706 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.220739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.220724 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19582221-42b6-43fe-86ac-c5f1f2aa9e77-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.221160 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.221132 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/19582221-42b6-43fe-86ac-c5f1f2aa9e77-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.221302 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.221278 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caaba1f6-5de3-45d3-bd70-c71692fd5877-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-dmrp2\" (UID: \"caaba1f6-5de3-45d3-bd70-c71692fd5877\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" Jul 06 20:07:22.221345 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.221331 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/41eb1a6e-35a2-4a3b-8459-06ad50a80756-node-exporter-tls\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.221470 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.221456 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/caaba1f6-5de3-45d3-bd70-c71692fd5877-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-dmrp2\" (UID: \"caaba1f6-5de3-45d3-bd70-c71692fd5877\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" Jul 06 20:07:22.227915 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.227869 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkfnp\" (UniqueName: \"kubernetes.io/projected/19582221-42b6-43fe-86ac-c5f1f2aa9e77-kube-api-access-wkfnp\") pod \"kube-state-metrics-7c9dbb8b7d-vpzgl\" (UID: \"19582221-42b6-43fe-86ac-c5f1f2aa9e77\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.228643 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.228624 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psdwf\" (UniqueName: \"kubernetes.io/projected/caaba1f6-5de3-45d3-bd70-c71692fd5877-kube-api-access-psdwf\") pod \"openshift-state-metrics-54cf96dfcf-dmrp2\" (UID: \"caaba1f6-5de3-45d3-bd70-c71692fd5877\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" Jul 06 20:07:22.228723 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.228662 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr6p4\" (UniqueName: \"kubernetes.io/projected/41eb1a6e-35a2-4a3b-8459-06ad50a80756-kube-api-access-fr6p4\") pod \"node-exporter-nsnl5\" (UID: \"41eb1a6e-35a2-4a3b-8459-06ad50a80756\") " pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.337408 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.337357 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" Jul 06 20:07:22.371169 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.371141 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nsnl5" Jul 06 20:07:22.387710 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:07:22.387686 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41eb1a6e_35a2_4a3b_8459_06ad50a80756.slice/crio-210b07b218a85eed4605ccd7b167befdbf698f782f6610f30ae567c7946a5770 WatchSource:0}: Error finding container 210b07b218a85eed4605ccd7b167befdbf698f782f6610f30ae567c7946a5770: Status 404 returned error can't find the container with id 210b07b218a85eed4605ccd7b167befdbf698f782f6610f30ae567c7946a5770 Jul 06 20:07:22.395744 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.395666 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" Jul 06 20:07:22.511666 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.511625 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2"] Jul 06 20:07:22.517135 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:07:22.517101 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaaba1f6_5de3_45d3_bd70_c71692fd5877.slice/crio-b2dbe1258f87790b1b1cc3e85ab8ff9d35962578c743b241b2358eeadd73ccc1 WatchSource:0}: Error finding container b2dbe1258f87790b1b1cc3e85ab8ff9d35962578c743b241b2358eeadd73ccc1: Status 404 returned error can't find the container with id b2dbe1258f87790b1b1cc3e85ab8ff9d35962578c743b241b2358eeadd73ccc1 Jul 06 20:07:22.556248 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:22.556220 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl"] Jul 06 20:07:22.560276 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:07:22.560246 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19582221_42b6_43fe_86ac_c5f1f2aa9e77.slice/crio-9f0200512ff1fc6f81c151a4e2770edea4bd5bac6a16625104b5112e82b6b42c WatchSource:0}: Error finding container 9f0200512ff1fc6f81c151a4e2770edea4bd5bac6a16625104b5112e82b6b42c: Status 404 returned error can't find the container with id 9f0200512ff1fc6f81c151a4e2770edea4bd5bac6a16625104b5112e82b6b42c Jul 06 20:07:23.078822 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.078785 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nsnl5" event={"ID":"41eb1a6e-35a2-4a3b-8459-06ad50a80756","Type":"ContainerStarted","Data":"210b07b218a85eed4605ccd7b167befdbf698f782f6610f30ae567c7946a5770"} Jul 06 20:07:23.080239 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.080213 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" event={"ID":"19582221-42b6-43fe-86ac-c5f1f2aa9e77","Type":"ContainerStarted","Data":"9f0200512ff1fc6f81c151a4e2770edea4bd5bac6a16625104b5112e82b6b42c"} Jul 06 20:07:23.082253 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.082226 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" event={"ID":"caaba1f6-5de3-45d3-bd70-c71692fd5877","Type":"ContainerStarted","Data":"34078c89bcd13a83b74be9a40f4db9bd3fd87c1bb271b88848b1c5164f400766"} Jul 06 20:07:23.082361 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.082285 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" event={"ID":"caaba1f6-5de3-45d3-bd70-c71692fd5877","Type":"ContainerStarted","Data":"f5c91da11490b4cafc60457cff59d0b07fb924133d7bf68e6037da3e8d63a807"} Jul 06 20:07:23.082361 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.082313 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" event={"ID":"caaba1f6-5de3-45d3-bd70-c71692fd5877","Type":"ContainerStarted","Data":"b2dbe1258f87790b1b1cc3e85ab8ff9d35962578c743b241b2358eeadd73ccc1"} Jul 06 20:07:23.117994 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.117964 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 20:07:23.122145 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.122126 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.130636 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.130610 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 20:07:23.131265 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.131244 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 20:07:23.131620 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.131596 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 20:07:23.132041 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.131790 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 20:07:23.132041 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.131825 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 20:07:23.132041 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.131830 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 20:07:23.132041 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.131921 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 20:07:23.132041 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.131946 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s4rvk\"" Jul 06 20:07:23.132041 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.131990 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 20:07:23.132384 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.132212 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 20:07:23.147289 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.147266 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 20:07:23.226755 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.226661 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f641b0d-9b3c-49a9-8455-6818dd958c8f-config-out\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.226755 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.226702 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.226991 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.226766 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f641b0d-9b3c-49a9-8455-6818dd958c8f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.226991 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.226903 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.226991 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.226976 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5f641b0d-9b3c-49a9-8455-6818dd958c8f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.227156 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.227050 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.227156 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.227079 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-web-config\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.227156 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.227134 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr8jq\" (UniqueName: \"kubernetes.io/projected/5f641b0d-9b3c-49a9-8455-6818dd958c8f-kube-api-access-nr8jq\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.227302 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.227158 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f641b0d-9b3c-49a9-8455-6818dd958c8f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.227302 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.227224 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-config-volume\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.227302 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.227263 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f641b0d-9b3c-49a9-8455-6818dd958c8f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.227537 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.227304 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.227537 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.227331 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.329159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.328252 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-config-volume\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.329159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.328303 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f641b0d-9b3c-49a9-8455-6818dd958c8f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.329159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.328338 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.329159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.328367 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.329159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.328404 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f641b0d-9b3c-49a9-8455-6818dd958c8f-config-out\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.329159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.328427 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.329159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.328468 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f641b0d-9b3c-49a9-8455-6818dd958c8f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.329159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.328506 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.329159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.328541 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5f641b0d-9b3c-49a9-8455-6818dd958c8f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.329159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.328569 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.329159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.328596 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-web-config\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.329159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.328638 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr8jq\" (UniqueName: \"kubernetes.io/projected/5f641b0d-9b3c-49a9-8455-6818dd958c8f-kube-api-access-nr8jq\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.329159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.328663 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f641b0d-9b3c-49a9-8455-6818dd958c8f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.329912 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.329384 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f641b0d-9b3c-49a9-8455-6818dd958c8f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.333725 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.333067 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f641b0d-9b3c-49a9-8455-6818dd958c8f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.333725 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.333529 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.335334 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.334898 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5f641b0d-9b3c-49a9-8455-6818dd958c8f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.336932 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.336266 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.337418 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.337375 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-config-volume\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.348886 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.348839 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.348994 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.348863 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f641b0d-9b3c-49a9-8455-6818dd958c8f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.349073 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.349053 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.349290 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.349239 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f641b0d-9b3c-49a9-8455-6818dd958c8f-config-out\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.349394 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.349351 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.351953 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.351931 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f641b0d-9b3c-49a9-8455-6818dd958c8f-web-config\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.352185 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.352133 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr8jq\" (UniqueName: \"kubernetes.io/projected/5f641b0d-9b3c-49a9-8455-6818dd958c8f-kube-api-access-nr8jq\") pod \"alertmanager-main-0\" (UID: \"5f641b0d-9b3c-49a9-8455-6818dd958c8f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:23.432787 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:23.432751 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 20:07:24.370494 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:24.370469 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 20:07:24.374220 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:07:24.374194 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f641b0d_9b3c_49a9_8455_6818dd958c8f.slice/crio-f77ecbe8d89434e832b025f98132a7a6216609f07450d632d7c0255e505b9306 WatchSource:0}: Error finding container f77ecbe8d89434e832b025f98132a7a6216609f07450d632d7c0255e505b9306: Status 404 returned error can't find the container with id f77ecbe8d89434e832b025f98132a7a6216609f07450d632d7c0255e505b9306 Jul 06 20:07:25.091820 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:25.091787 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" event={"ID":"caaba1f6-5de3-45d3-bd70-c71692fd5877","Type":"ContainerStarted","Data":"22337bfe97ec8a43da1863b55b1ea7539a8679a33fd4af08e2411d1bbcee4795"} Jul 06 20:07:25.093156 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:25.093129 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f641b0d-9b3c-49a9-8455-6818dd958c8f","Type":"ContainerStarted","Data":"f77ecbe8d89434e832b025f98132a7a6216609f07450d632d7c0255e505b9306"} Jul 06 20:07:25.094752 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:25.094724 2583 generic.go:356] "Generic (PLEG): container finished" podID="41eb1a6e-35a2-4a3b-8459-06ad50a80756" containerID="83c7d5499192cbf0ddcdcc178c78200c85b2046069f68fa2b6620140890702f0" exitCode=0 Jul 06 20:07:25.094848 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:25.094806 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nsnl5" event={"ID":"41eb1a6e-35a2-4a3b-8459-06ad50a80756","Type":"ContainerDied","Data":"83c7d5499192cbf0ddcdcc178c78200c85b2046069f68fa2b6620140890702f0"} Jul 06 20:07:25.096958 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:25.096934 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" event={"ID":"19582221-42b6-43fe-86ac-c5f1f2aa9e77","Type":"ContainerStarted","Data":"8bdc94b80860e2ff1749110c065cb74d0a32ed41882e9615a0882a59262eedeb"} Jul 06 20:07:25.097040 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:25.096979 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" event={"ID":"19582221-42b6-43fe-86ac-c5f1f2aa9e77","Type":"ContainerStarted","Data":"236c25deac5d1c3b7f319957cdef0a8afec3feb6346d611e73f37114b3df581c"} Jul 06 20:07:25.097040 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:25.097009 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" event={"ID":"19582221-42b6-43fe-86ac-c5f1f2aa9e77","Type":"ContainerStarted","Data":"bd3eaa2bd0a890afce738979351d95722281f505716afa04c8cabf2767af1b1c"} Jul 06 20:07:25.118251 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:25.118213 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmrp2" podStartSLOduration=2.5724859650000003 podStartE2EDuration="4.118195857s" totalImagesPullingTime="1.545709892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:07:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:07:25.117013436 +0000 UTC m=+98.243925552" watchObservedRunningTime="2026-07-06 20:07:25.118195857 +0000 UTC m=+98.245107962" Jul 06 20:07:25.164503 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:25.164457 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vpzgl" podStartSLOduration=1.510668881 podStartE2EDuration="3.16443782s" totalImagesPullingTime="1.653768939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:07:25.16356772 +0000 UTC m=+98.290479824" watchObservedRunningTime="2026-07-06 20:07:25.16443782 +0000 UTC m=+98.291349926" Jul 06 20:07:25.932789 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:25.932717 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-spktr" Jul 06 20:07:26.107669 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:26.107638 2583 generic.go:356] "Generic (PLEG): container finished" podID="5f641b0d-9b3c-49a9-8455-6818dd958c8f" containerID="520bdc3441a5e485b2f4c48f200d68fc44dab0493be285b4335747ee80e9ac7d" exitCode=0 Jul 06 20:07:26.107845 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:26.107734 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f641b0d-9b3c-49a9-8455-6818dd958c8f","Type":"ContainerDied","Data":"520bdc3441a5e485b2f4c48f200d68fc44dab0493be285b4335747ee80e9ac7d"} Jul 06 20:07:26.109922 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:26.109900 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nsnl5" event={"ID":"41eb1a6e-35a2-4a3b-8459-06ad50a80756","Type":"ContainerStarted","Data":"a75ce359a82ff3fb9522a29958a77ecf5e67465494a115b0d424d7dd84b4dd70"} Jul 06 20:07:26.110005 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:26.109949 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nsnl5" event={"ID":"41eb1a6e-35a2-4a3b-8459-06ad50a80756","Type":"ContainerStarted","Data":"5f952eae25c5234687f2fe6b83dda9b5e0b4063eccdb04dd57388c35dd8028b8"} Jul 06 20:07:26.177598 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:26.177551 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nsnl5" podStartSLOduration=2.28362541 podStartE2EDuration="4.177537516s" totalImagesPullingTime="1.820228293s" totalInitContainerRuntime="73.683813ms" isStatefulPod=true podCreationTimestamp="2026-07-06 20:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:07:26.177281526 +0000 UTC m=+99.304193629" watchObservedRunningTime="2026-07-06 20:07:26.177537516 +0000 UTC m=+99.304449649" Jul 06 20:07:28.121184 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:28.121158 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f641b0d-9b3c-49a9-8455-6818dd958c8f","Type":"ContainerStarted","Data":"ffbe1f3157f7dd75a7222d8f57097af569a4fc8ee4884bc463c93f53563acb1e"} Jul 06 20:07:28.121626 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:28.121219 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f641b0d-9b3c-49a9-8455-6818dd958c8f","Type":"ContainerStarted","Data":"d7a80208fb26b24806df979b846d6bc0f5339549ec3194c45d5c618b8b36f219"} Jul 06 20:07:29.134296 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:29.134255 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f641b0d-9b3c-49a9-8455-6818dd958c8f","Type":"ContainerStarted","Data":"9e907100b9b44945bd8cab80bcc97d8f725050cbe82efe28b1d0fa47f7a9813e"} Jul 06 20:07:29.134748 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:29.134371 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f641b0d-9b3c-49a9-8455-6818dd958c8f","Type":"ContainerStarted","Data":"d09122ffd660571b517ef31265a6c6d2a3be5944f95e300e2610675666adfd8e"} Jul 06 20:07:29.134748 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:29.134413 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f641b0d-9b3c-49a9-8455-6818dd958c8f","Type":"ContainerStarted","Data":"cd72f1ac8615988173111836eed66fa748d8af3456ad310278a2af602b422d7e"} Jul 06 20:07:30.141340 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:30.141269 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f641b0d-9b3c-49a9-8455-6818dd958c8f","Type":"ContainerStarted","Data":"1bc538cdb42207be7e53ba90d8ed9dd8dd5a3d3016af36db9046d26b9e69c185"} Jul 06 20:07:30.176729 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:30.176679 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.8061624910000003 podStartE2EDuration="7.176663849s" totalImagesPullingTime="4.355619171s" totalInitContainerRuntime="14.882187ms" isStatefulPod=false podCreationTimestamp="2026-07-06 20:07:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:07:30.175394771 +0000 UTC m=+103.302306886" watchObservedRunningTime="2026-07-06 20:07:30.176663849 +0000 UTC m=+103.303575952" Jul 06 20:07:43.513955 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.513916 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-58d6568c64-ngz78" podUID="03cefcbc-3546-4aea-bcac-33036307b703" containerName="registry" containerID="cri-o://f1432994987f67990e81f54dc289c605cfbd417527b025f5205f15bbffeedd94" gracePeriod=30 Jul 06 20:07:43.759889 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.759850 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:07:43.787999 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.787936 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls\") pod \"03cefcbc-3546-4aea-bcac-33036307b703\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " Jul 06 20:07:43.788124 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.788007 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjpt2\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-kube-api-access-tjpt2\") pod \"03cefcbc-3546-4aea-bcac-33036307b703\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " Jul 06 20:07:43.788124 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.788032 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/03cefcbc-3546-4aea-bcac-33036307b703-trusted-ca\") pod \"03cefcbc-3546-4aea-bcac-33036307b703\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " Jul 06 20:07:43.788124 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.788079 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/03cefcbc-3546-4aea-bcac-33036307b703-ca-trust-extracted\") pod \"03cefcbc-3546-4aea-bcac-33036307b703\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " Jul 06 20:07:43.788124 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.788105 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-bound-sa-token\") pod \"03cefcbc-3546-4aea-bcac-33036307b703\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " Jul 06 20:07:43.788319 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.788139 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/03cefcbc-3546-4aea-bcac-33036307b703-image-registry-private-configuration\") pod \"03cefcbc-3546-4aea-bcac-33036307b703\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " Jul 06 20:07:43.788319 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.788170 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/03cefcbc-3546-4aea-bcac-33036307b703-installation-pull-secrets\") pod \"03cefcbc-3546-4aea-bcac-33036307b703\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " Jul 06 20:07:43.788319 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.788221 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/03cefcbc-3546-4aea-bcac-33036307b703-registry-certificates\") pod \"03cefcbc-3546-4aea-bcac-33036307b703\" (UID: \"03cefcbc-3546-4aea-bcac-33036307b703\") " Jul 06 20:07:43.788578 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.788548 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03cefcbc-3546-4aea-bcac-33036307b703-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "03cefcbc-3546-4aea-bcac-33036307b703" (UID: "03cefcbc-3546-4aea-bcac-33036307b703"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:07:43.789298 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.788866 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03cefcbc-3546-4aea-bcac-33036307b703-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "03cefcbc-3546-4aea-bcac-33036307b703" (UID: "03cefcbc-3546-4aea-bcac-33036307b703"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:07:43.791359 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.791334 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-kube-api-access-tjpt2" (OuterVolumeSpecName: "kube-api-access-tjpt2") pod "03cefcbc-3546-4aea-bcac-33036307b703" (UID: "03cefcbc-3546-4aea-bcac-33036307b703"). InnerVolumeSpecName "kube-api-access-tjpt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:07:43.791518 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.791488 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "03cefcbc-3546-4aea-bcac-33036307b703" (UID: "03cefcbc-3546-4aea-bcac-33036307b703"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:07:43.791629 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.791606 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "03cefcbc-3546-4aea-bcac-33036307b703" (UID: "03cefcbc-3546-4aea-bcac-33036307b703"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:07:43.791730 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.791704 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03cefcbc-3546-4aea-bcac-33036307b703-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "03cefcbc-3546-4aea-bcac-33036307b703" (UID: "03cefcbc-3546-4aea-bcac-33036307b703"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:07:43.791843 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.791822 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03cefcbc-3546-4aea-bcac-33036307b703-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "03cefcbc-3546-4aea-bcac-33036307b703" (UID: "03cefcbc-3546-4aea-bcac-33036307b703"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:07:43.797806 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.797779 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03cefcbc-3546-4aea-bcac-33036307b703-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "03cefcbc-3546-4aea-bcac-33036307b703" (UID: "03cefcbc-3546-4aea-bcac-33036307b703"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:07:43.889330 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.889285 2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/03cefcbc-3546-4aea-bcac-33036307b703-ca-trust-extracted\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:07:43.889330 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.889323 2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-bound-sa-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:07:43.889330 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.889338 2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/03cefcbc-3546-4aea-bcac-33036307b703-image-registry-private-configuration\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:07:43.889566 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.889350 2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/03cefcbc-3546-4aea-bcac-33036307b703-installation-pull-secrets\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:07:43.889566 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.889363 2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/03cefcbc-3546-4aea-bcac-33036307b703-registry-certificates\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:07:43.889566 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.889374 2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-registry-tls\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:07:43.889566 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.889385 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjpt2\" (UniqueName: \"kubernetes.io/projected/03cefcbc-3546-4aea-bcac-33036307b703-kube-api-access-tjpt2\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:07:43.889566 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:43.889394 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/03cefcbc-3546-4aea-bcac-33036307b703-trusted-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:07:44.196530 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:44.196442 2583 generic.go:356] "Generic (PLEG): container finished" podID="03cefcbc-3546-4aea-bcac-33036307b703" containerID="f1432994987f67990e81f54dc289c605cfbd417527b025f5205f15bbffeedd94" exitCode=0 Jul 06 20:07:44.196530 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:44.196488 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58d6568c64-ngz78" Jul 06 20:07:44.196530 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:44.196493 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58d6568c64-ngz78" event={"ID":"03cefcbc-3546-4aea-bcac-33036307b703","Type":"ContainerDied","Data":"f1432994987f67990e81f54dc289c605cfbd417527b025f5205f15bbffeedd94"} Jul 06 20:07:44.196733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:44.196543 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58d6568c64-ngz78" event={"ID":"03cefcbc-3546-4aea-bcac-33036307b703","Type":"ContainerDied","Data":"aedc500523ec9894fcc29bb70b267f291628d077dfaa8eb08a249c88b842e274"} Jul 06 20:07:44.196733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:44.196569 2583 scope.go:117] "RemoveContainer" containerID="f1432994987f67990e81f54dc289c605cfbd417527b025f5205f15bbffeedd94" Jul 06 20:07:44.205322 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:44.205304 2583 scope.go:117] "RemoveContainer" containerID="f1432994987f67990e81f54dc289c605cfbd417527b025f5205f15bbffeedd94" Jul 06 20:07:44.205576 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:07:44.205555 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1432994987f67990e81f54dc289c605cfbd417527b025f5205f15bbffeedd94\": container with ID starting with f1432994987f67990e81f54dc289c605cfbd417527b025f5205f15bbffeedd94 not found: ID does not exist" containerID="f1432994987f67990e81f54dc289c605cfbd417527b025f5205f15bbffeedd94" Jul 06 20:07:44.205640 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:44.205585 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1432994987f67990e81f54dc289c605cfbd417527b025f5205f15bbffeedd94"} err="failed to get container status \"f1432994987f67990e81f54dc289c605cfbd417527b025f5205f15bbffeedd94\": rpc error: code = NotFound desc = could not find container \"f1432994987f67990e81f54dc289c605cfbd417527b025f5205f15bbffeedd94\": container with ID starting with f1432994987f67990e81f54dc289c605cfbd417527b025f5205f15bbffeedd94 not found: ID does not exist" Jul 06 20:07:44.221555 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:44.221532 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-58d6568c64-ngz78"] Jul 06 20:07:44.226914 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:44.226891 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-58d6568c64-ngz78"] Jul 06 20:07:45.483998 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:45.483968 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03cefcbc-3546-4aea-bcac-33036307b703" path="/var/lib/kubelet/pods/03cefcbc-3546-4aea-bcac-33036307b703/volumes" Jul 06 20:07:45.727894 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:45.727849 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68497d5b9b-d298g"] Jul 06 20:07:48.213191 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:48.213111 2583 generic.go:356] "Generic (PLEG): container finished" podID="c91495de-ffbb-4d96-a6a8-5511288f7cd0" containerID="6f2908bcf08f94aa9edd7b2bacfcd896049687de32ac4e26c504aa1042c3e043" exitCode=0 Jul 06 20:07:48.213544 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:48.213183 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" event={"ID":"c91495de-ffbb-4d96-a6a8-5511288f7cd0","Type":"ContainerDied","Data":"6f2908bcf08f94aa9edd7b2bacfcd896049687de32ac4e26c504aa1042c3e043"} Jul 06 20:07:48.213544 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:48.213527 2583 scope.go:117] "RemoveContainer" containerID="6f2908bcf08f94aa9edd7b2bacfcd896049687de32ac4e26c504aa1042c3e043" Jul 06 20:07:49.218286 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:49.218257 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nln8b" event={"ID":"c91495de-ffbb-4d96-a6a8-5511288f7cd0","Type":"ContainerStarted","Data":"1d5804c0447f24263c23600713218115f429fc27af34a07df3739274a6ef6b55"} Jul 06 20:07:54.243026 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:54.242992 2583 generic.go:356] "Generic (PLEG): container finished" podID="b359f2f9-4eb6-47f8-90cf-f096ba41e852" containerID="3f09e421716c6d7150cb388e653669f7ac348f25ffddba9300667a0041b14f7a" exitCode=0 Jul 06 20:07:54.243412 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:54.243061 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" event={"ID":"b359f2f9-4eb6-47f8-90cf-f096ba41e852","Type":"ContainerDied","Data":"3f09e421716c6d7150cb388e653669f7ac348f25ffddba9300667a0041b14f7a"} Jul 06 20:07:54.243412 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:54.243370 2583 scope.go:117] "RemoveContainer" containerID="3f09e421716c6d7150cb388e653669f7ac348f25ffddba9300667a0041b14f7a" Jul 06 20:07:55.249576 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:07:55.249543 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hbc8g" event={"ID":"b359f2f9-4eb6-47f8-90cf-f096ba41e852","Type":"ContainerStarted","Data":"20c18c7ddf54bc53677668181f3968d10945663c0cca500d6c09af528cfe9350"} Jul 06 20:08:03.287264 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:03.287231 2583 generic.go:356] "Generic (PLEG): container finished" podID="db968f0a-d281-4b2b-8eaf-285d77c876b2" containerID="9d3980867bb15347bb610bd05160088b81027cca92aabfbc9e10e1006045f377" exitCode=0 Jul 06 20:08:03.287650 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:03.287272 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" event={"ID":"db968f0a-d281-4b2b-8eaf-285d77c876b2","Type":"ContainerDied","Data":"9d3980867bb15347bb610bd05160088b81027cca92aabfbc9e10e1006045f377"} Jul 06 20:08:03.287650 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:03.287580 2583 scope.go:117] "RemoveContainer" containerID="9d3980867bb15347bb610bd05160088b81027cca92aabfbc9e10e1006045f377" Jul 06 20:08:04.293105 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:04.293072 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-54f2g" event={"ID":"db968f0a-d281-4b2b-8eaf-285d77c876b2","Type":"ContainerStarted","Data":"ca143471171f9cb3e729ade55f1293df2f9ce7b3abad7b9fc35b726f8f3548d5"} Jul 06 20:08:10.748805 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:10.748750 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-68497d5b9b-d298g" podUID="c37deaf8-205f-428b-a120-0afffdabdbcd" containerName="console" containerID="cri-o://e9631f7f7124f19190c790b49216f5af8ea52360cbc1d7666b727ad8f6b3f884" gracePeriod=15 Jul 06 20:08:11.000832 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.000777 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:08:11.115340 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.115302 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c37deaf8-205f-428b-a120-0afffdabdbcd-console-serving-cert\") pod \"c37deaf8-205f-428b-a120-0afffdabdbcd\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " Jul 06 20:08:11.115496 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.115396 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-console-config\") pod \"c37deaf8-205f-428b-a120-0afffdabdbcd\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " Jul 06 20:08:11.115570 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.115506 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-oauth-serving-cert\") pod \"c37deaf8-205f-428b-a120-0afffdabdbcd\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " Jul 06 20:08:11.115570 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.115544 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-trusted-ca-bundle\") pod \"c37deaf8-205f-428b-a120-0afffdabdbcd\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " Jul 06 20:08:11.115670 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.115569 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-service-ca\") pod \"c37deaf8-205f-428b-a120-0afffdabdbcd\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " Jul 06 20:08:11.115670 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.115612 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6m5g\" (UniqueName: \"kubernetes.io/projected/c37deaf8-205f-428b-a120-0afffdabdbcd-kube-api-access-r6m5g\") pod \"c37deaf8-205f-428b-a120-0afffdabdbcd\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " Jul 06 20:08:11.115670 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.115641 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-console-config" (OuterVolumeSpecName: "console-config") pod "c37deaf8-205f-428b-a120-0afffdabdbcd" (UID: "c37deaf8-205f-428b-a120-0afffdabdbcd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:08:11.115670 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.115648 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c37deaf8-205f-428b-a120-0afffdabdbcd-console-oauth-config\") pod \"c37deaf8-205f-428b-a120-0afffdabdbcd\" (UID: \"c37deaf8-205f-428b-a120-0afffdabdbcd\") " Jul 06 20:08:11.115961 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.115940 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c37deaf8-205f-428b-a120-0afffdabdbcd" (UID: "c37deaf8-205f-428b-a120-0afffdabdbcd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:08:11.116040 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.115955 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-service-ca" (OuterVolumeSpecName: "service-ca") pod "c37deaf8-205f-428b-a120-0afffdabdbcd" (UID: "c37deaf8-205f-428b-a120-0afffdabdbcd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:08:11.116040 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.115962 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c37deaf8-205f-428b-a120-0afffdabdbcd" (UID: "c37deaf8-205f-428b-a120-0afffdabdbcd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:08:11.116131 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.116113 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-console-config\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:08:11.116167 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.116136 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-oauth-serving-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:08:11.116167 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.116150 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-trusted-ca-bundle\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:08:11.116240 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.116164 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c37deaf8-205f-428b-a120-0afffdabdbcd-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:08:11.117474 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.117448 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c37deaf8-205f-428b-a120-0afffdabdbcd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c37deaf8-205f-428b-a120-0afffdabdbcd" (UID: "c37deaf8-205f-428b-a120-0afffdabdbcd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:08:11.117659 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.117632 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c37deaf8-205f-428b-a120-0afffdabdbcd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c37deaf8-205f-428b-a120-0afffdabdbcd" (UID: "c37deaf8-205f-428b-a120-0afffdabdbcd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:08:11.117754 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.117721 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c37deaf8-205f-428b-a120-0afffdabdbcd-kube-api-access-r6m5g" (OuterVolumeSpecName: "kube-api-access-r6m5g") pod "c37deaf8-205f-428b-a120-0afffdabdbcd" (UID: "c37deaf8-205f-428b-a120-0afffdabdbcd"). InnerVolumeSpecName "kube-api-access-r6m5g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:08:11.216845 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.216815 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c37deaf8-205f-428b-a120-0afffdabdbcd-console-serving-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:08:11.216845 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.216845 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6m5g\" (UniqueName: \"kubernetes.io/projected/c37deaf8-205f-428b-a120-0afffdabdbcd-kube-api-access-r6m5g\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:08:11.216845 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.216854 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c37deaf8-205f-428b-a120-0afffdabdbcd-console-oauth-config\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:08:11.322704 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.322648 2583 generic.go:356] "Generic (PLEG): container finished" podID="c37deaf8-205f-428b-a120-0afffdabdbcd" containerID="e9631f7f7124f19190c790b49216f5af8ea52360cbc1d7666b727ad8f6b3f884" exitCode=0 Jul 06 20:08:11.322704 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.322690 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68497d5b9b-d298g" Jul 06 20:08:11.322814 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.322729 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68497d5b9b-d298g" event={"ID":"c37deaf8-205f-428b-a120-0afffdabdbcd","Type":"ContainerDied","Data":"e9631f7f7124f19190c790b49216f5af8ea52360cbc1d7666b727ad8f6b3f884"} Jul 06 20:08:11.322814 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.322777 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68497d5b9b-d298g" event={"ID":"c37deaf8-205f-428b-a120-0afffdabdbcd","Type":"ContainerDied","Data":"85bcc3502b8b1955cda9c8e95467213562f361a71a95a4a5605da36f37b3cd46"} Jul 06 20:08:11.322814 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.322800 2583 scope.go:117] "RemoveContainer" containerID="e9631f7f7124f19190c790b49216f5af8ea52360cbc1d7666b727ad8f6b3f884" Jul 06 20:08:11.331174 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.331097 2583 scope.go:117] "RemoveContainer" containerID="e9631f7f7124f19190c790b49216f5af8ea52360cbc1d7666b727ad8f6b3f884" Jul 06 20:08:11.331419 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:08:11.331399 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9631f7f7124f19190c790b49216f5af8ea52360cbc1d7666b727ad8f6b3f884\": container with ID starting with e9631f7f7124f19190c790b49216f5af8ea52360cbc1d7666b727ad8f6b3f884 not found: ID does not exist" containerID="e9631f7f7124f19190c790b49216f5af8ea52360cbc1d7666b727ad8f6b3f884" Jul 06 20:08:11.331506 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.331425 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9631f7f7124f19190c790b49216f5af8ea52360cbc1d7666b727ad8f6b3f884"} err="failed to get container status \"e9631f7f7124f19190c790b49216f5af8ea52360cbc1d7666b727ad8f6b3f884\": rpc error: code = NotFound desc = could not find container \"e9631f7f7124f19190c790b49216f5af8ea52360cbc1d7666b727ad8f6b3f884\": container with ID starting with e9631f7f7124f19190c790b49216f5af8ea52360cbc1d7666b727ad8f6b3f884 not found: ID does not exist" Jul 06 20:08:11.350763 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.348627 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68497d5b9b-d298g"] Jul 06 20:08:11.352542 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.352520 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68497d5b9b-d298g"] Jul 06 20:08:11.483844 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:11.483823 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c37deaf8-205f-428b-a120-0afffdabdbcd" path="/var/lib/kubelet/pods/c37deaf8-205f-428b-a120-0afffdabdbcd/volumes" Jul 06 20:08:48.995130 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:48.995095 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b4dd5966-rqgrx"] Jul 06 20:08:48.995617 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:48.995535 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03cefcbc-3546-4aea-bcac-33036307b703" containerName="registry" Jul 06 20:08:48.995617 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:48.995564 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="03cefcbc-3546-4aea-bcac-33036307b703" containerName="registry" Jul 06 20:08:48.995617 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:48.995602 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c37deaf8-205f-428b-a120-0afffdabdbcd" containerName="console" Jul 06 20:08:48.995617 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:48.995609 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c37deaf8-205f-428b-a120-0afffdabdbcd" containerName="console" Jul 06 20:08:48.995832 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:48.995678 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="03cefcbc-3546-4aea-bcac-33036307b703" containerName="registry" Jul 06 20:08:48.995832 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:48.995696 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c37deaf8-205f-428b-a120-0afffdabdbcd" containerName="console" Jul 06 20:08:48.999813 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:48.999791 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.003257 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.003236 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 20:08:49.003360 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.003264 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 20:08:49.004123 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.004105 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5gqqx\"" Jul 06 20:08:49.004435 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.004413 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 20:08:49.004552 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.004457 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 20:08:49.004552 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.004476 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 20:08:49.005419 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.005401 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 20:08:49.005517 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.005451 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 20:08:49.008720 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.008702 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 20:08:49.011200 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.011178 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b4dd5966-rqgrx"] Jul 06 20:08:49.081485 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.081443 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-oauth-config\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.081623 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.081522 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-serving-cert\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.081623 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.081612 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26vnj\" (UniqueName: \"kubernetes.io/projected/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-kube-api-access-26vnj\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.081745 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.081663 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-config\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.081745 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.081693 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-trusted-ca-bundle\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.081745 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.081732 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-service-ca\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.081864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.081766 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-oauth-serving-cert\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.182974 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.182951 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-service-ca\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.183110 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.182986 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-oauth-serving-cert\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.183110 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.183004 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-oauth-config\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.183110 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.183022 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-serving-cert\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.183282 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.183162 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26vnj\" (UniqueName: \"kubernetes.io/projected/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-kube-api-access-26vnj\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.183282 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.183219 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-config\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.183282 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.183247 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-trusted-ca-bundle\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.183780 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.183754 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-service-ca\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.183893 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.183822 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-oauth-serving-cert\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.183969 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.183909 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-config\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.184165 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.184145 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-trusted-ca-bundle\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.185593 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.185566 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-oauth-config\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.185668 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.185631 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-serving-cert\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.193838 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.193817 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26vnj\" (UniqueName: \"kubernetes.io/projected/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-kube-api-access-26vnj\") pod \"console-6b4dd5966-rqgrx\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.311038 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.310987 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:49.449866 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.449839 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b4dd5966-rqgrx"] Jul 06 20:08:49.452603 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:08:49.452580 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10f4b0d9_d85d_4bdf_8ffe_fccc93237b2a.slice/crio-7e10f1ac9c25db791f0f622b61d5cf1191cfebb7bc1aa15846cee988ab3a5207 WatchSource:0}: Error finding container 7e10f1ac9c25db791f0f622b61d5cf1191cfebb7bc1aa15846cee988ab3a5207: Status 404 returned error can't find the container with id 7e10f1ac9c25db791f0f622b61d5cf1191cfebb7bc1aa15846cee988ab3a5207 Jul 06 20:08:49.478407 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:49.478381 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b4dd5966-rqgrx" event={"ID":"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a","Type":"ContainerStarted","Data":"7e10f1ac9c25db791f0f622b61d5cf1191cfebb7bc1aa15846cee988ab3a5207"} Jul 06 20:08:50.483999 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:50.483970 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b4dd5966-rqgrx" event={"ID":"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a","Type":"ContainerStarted","Data":"39b13291a0e89ff1716ab57626e36c5d2174042edca11a116cd390c4289ae2a9"} Jul 06 20:08:50.507160 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:50.507112 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b4dd5966-rqgrx" podStartSLOduration=2.507096691 podStartE2EDuration="2.507096691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:08:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:08:50.504935668 +0000 UTC m=+183.631847773" watchObservedRunningTime="2026-07-06 20:08:50.507096691 +0000 UTC m=+183.634008795" Jul 06 20:08:59.311643 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:59.311604 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:59.312126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:59.311669 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:59.315861 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:59.315843 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:08:59.518616 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:08:59.518592 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:09:15.118075 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.118042 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6"] Jul 06 20:09:15.121490 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.121469 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" Jul 06 20:09:15.123978 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.123957 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 20:09:15.124907 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.124867 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 20:09:15.124983 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.124938 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nl5wp\"" Jul 06 20:09:15.133159 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.133139 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6"] Jul 06 20:09:15.170312 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.170291 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1eef98d-d66a-4901-b27d-f66deb614df8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6\" (UID: \"e1eef98d-d66a-4901-b27d-f66deb614df8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" Jul 06 20:09:15.170406 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.170328 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1eef98d-d66a-4901-b27d-f66deb614df8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6\" (UID: \"e1eef98d-d66a-4901-b27d-f66deb614df8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" Jul 06 20:09:15.170406 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.170366 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2llm\" (UniqueName: \"kubernetes.io/projected/e1eef98d-d66a-4901-b27d-f66deb614df8-kube-api-access-k2llm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6\" (UID: \"e1eef98d-d66a-4901-b27d-f66deb614df8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" Jul 06 20:09:15.271108 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.271083 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1eef98d-d66a-4901-b27d-f66deb614df8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6\" (UID: \"e1eef98d-d66a-4901-b27d-f66deb614df8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" Jul 06 20:09:15.271204 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.271120 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1eef98d-d66a-4901-b27d-f66deb614df8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6\" (UID: \"e1eef98d-d66a-4901-b27d-f66deb614df8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" Jul 06 20:09:15.271335 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.271316 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2llm\" (UniqueName: \"kubernetes.io/projected/e1eef98d-d66a-4901-b27d-f66deb614df8-kube-api-access-k2llm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6\" (UID: \"e1eef98d-d66a-4901-b27d-f66deb614df8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" Jul 06 20:09:15.271422 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.271407 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1eef98d-d66a-4901-b27d-f66deb614df8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6\" (UID: \"e1eef98d-d66a-4901-b27d-f66deb614df8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" Jul 06 20:09:15.271466 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.271433 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1eef98d-d66a-4901-b27d-f66deb614df8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6\" (UID: \"e1eef98d-d66a-4901-b27d-f66deb614df8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" Jul 06 20:09:15.283090 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.283070 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2llm\" (UniqueName: \"kubernetes.io/projected/e1eef98d-d66a-4901-b27d-f66deb614df8-kube-api-access-k2llm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6\" (UID: \"e1eef98d-d66a-4901-b27d-f66deb614df8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" Jul 06 20:09:15.430353 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.430303 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" Jul 06 20:09:15.561609 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.561585 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6"] Jul 06 20:09:15.563455 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:09:15.563426 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1eef98d_d66a_4901_b27d_f66deb614df8.slice/crio-966d3f082f08062c94bf73ccf09b6749b05612ca4801b21b8a3336174629cd4a WatchSource:0}: Error finding container 966d3f082f08062c94bf73ccf09b6749b05612ca4801b21b8a3336174629cd4a: Status 404 returned error can't find the container with id 966d3f082f08062c94bf73ccf09b6749b05612ca4801b21b8a3336174629cd4a Jul 06 20:09:15.573092 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:15.573066 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" event={"ID":"e1eef98d-d66a-4901-b27d-f66deb614df8","Type":"ContainerStarted","Data":"966d3f082f08062c94bf73ccf09b6749b05612ca4801b21b8a3336174629cd4a"} Jul 06 20:09:25.614646 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:25.614605 2583 generic.go:356] "Generic (PLEG): container finished" podID="e1eef98d-d66a-4901-b27d-f66deb614df8" containerID="52aa39ba840eca6a1c6a2922f0f0df7a66984f9c54cd62e86cea256beec7ae11" exitCode=0 Jul 06 20:09:25.614995 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:25.614676 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" event={"ID":"e1eef98d-d66a-4901-b27d-f66deb614df8","Type":"ContainerDied","Data":"52aa39ba840eca6a1c6a2922f0f0df7a66984f9c54cd62e86cea256beec7ae11"} Jul 06 20:09:27.625601 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:27.625569 2583 generic.go:356] "Generic (PLEG): container finished" podID="e1eef98d-d66a-4901-b27d-f66deb614df8" containerID="da5e748eec67d958c30a8dd0d19e5d28690293fd4f193be6da3257bd52ff4407" exitCode=0 Jul 06 20:09:27.625914 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:27.625607 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" event={"ID":"e1eef98d-d66a-4901-b27d-f66deb614df8","Type":"ContainerDied","Data":"da5e748eec67d958c30a8dd0d19e5d28690293fd4f193be6da3257bd52ff4407"} Jul 06 20:09:37.672262 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:37.672230 2583 generic.go:356] "Generic (PLEG): container finished" podID="e1eef98d-d66a-4901-b27d-f66deb614df8" containerID="f2049dfaf86230ea5abac8a7d228886ec1dfb559a0d7f5ed16a66efb01d62c68" exitCode=0 Jul 06 20:09:37.672657 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:37.672274 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" event={"ID":"e1eef98d-d66a-4901-b27d-f66deb614df8","Type":"ContainerDied","Data":"f2049dfaf86230ea5abac8a7d228886ec1dfb559a0d7f5ed16a66efb01d62c68"} Jul 06 20:09:38.802813 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:38.802788 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" Jul 06 20:09:38.964235 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:38.964174 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2llm\" (UniqueName: \"kubernetes.io/projected/e1eef98d-d66a-4901-b27d-f66deb614df8-kube-api-access-k2llm\") pod \"e1eef98d-d66a-4901-b27d-f66deb614df8\" (UID: \"e1eef98d-d66a-4901-b27d-f66deb614df8\") " Jul 06 20:09:38.964235 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:38.964228 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1eef98d-d66a-4901-b27d-f66deb614df8-util\") pod \"e1eef98d-d66a-4901-b27d-f66deb614df8\" (UID: \"e1eef98d-d66a-4901-b27d-f66deb614df8\") " Jul 06 20:09:38.964391 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:38.964249 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1eef98d-d66a-4901-b27d-f66deb614df8-bundle\") pod \"e1eef98d-d66a-4901-b27d-f66deb614df8\" (UID: \"e1eef98d-d66a-4901-b27d-f66deb614df8\") " Jul 06 20:09:38.964884 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:38.964853 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1eef98d-d66a-4901-b27d-f66deb614df8-bundle" (OuterVolumeSpecName: "bundle") pod "e1eef98d-d66a-4901-b27d-f66deb614df8" (UID: "e1eef98d-d66a-4901-b27d-f66deb614df8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:38.966288 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:38.966266 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1eef98d-d66a-4901-b27d-f66deb614df8-kube-api-access-k2llm" (OuterVolumeSpecName: "kube-api-access-k2llm") pod "e1eef98d-d66a-4901-b27d-f66deb614df8" (UID: "e1eef98d-d66a-4901-b27d-f66deb614df8"). InnerVolumeSpecName "kube-api-access-k2llm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:09:38.968383 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:38.968361 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1eef98d-d66a-4901-b27d-f66deb614df8-util" (OuterVolumeSpecName: "util") pod "e1eef98d-d66a-4901-b27d-f66deb614df8" (UID: "e1eef98d-d66a-4901-b27d-f66deb614df8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:39.065084 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:39.065058 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2llm\" (UniqueName: \"kubernetes.io/projected/e1eef98d-d66a-4901-b27d-f66deb614df8-kube-api-access-k2llm\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:09:39.065084 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:39.065079 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1eef98d-d66a-4901-b27d-f66deb614df8-util\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:09:39.065213 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:39.065089 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1eef98d-d66a-4901-b27d-f66deb614df8-bundle\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:09:39.681355 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:39.681329 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" Jul 06 20:09:39.681501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:39.681322 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d89w6" event={"ID":"e1eef98d-d66a-4901-b27d-f66deb614df8","Type":"ContainerDied","Data":"966d3f082f08062c94bf73ccf09b6749b05612ca4801b21b8a3336174629cd4a"} Jul 06 20:09:39.681501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:39.681459 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="966d3f082f08062c94bf73ccf09b6749b05612ca4801b21b8a3336174629cd4a" Jul 06 20:09:47.018075 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.018044 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qvkzx"] Jul 06 20:09:47.018502 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.018340 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1eef98d-d66a-4901-b27d-f66deb614df8" containerName="util" Jul 06 20:09:47.018502 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.018351 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1eef98d-d66a-4901-b27d-f66deb614df8" containerName="util" Jul 06 20:09:47.018502 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.018359 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1eef98d-d66a-4901-b27d-f66deb614df8" containerName="extract" Jul 06 20:09:47.018502 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.018364 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1eef98d-d66a-4901-b27d-f66deb614df8" containerName="extract" Jul 06 20:09:47.018502 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.018375 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1eef98d-d66a-4901-b27d-f66deb614df8" containerName="pull" Jul 06 20:09:47.018502 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.018382 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1eef98d-d66a-4901-b27d-f66deb614df8" containerName="pull" Jul 06 20:09:47.018502 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.018446 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1eef98d-d66a-4901-b27d-f66deb614df8" containerName="extract" Jul 06 20:09:47.021779 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.021762 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qvkzx" Jul 06 20:09:47.024951 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.024930 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 20:09:47.032011 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.031992 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-ws6wn\"" Jul 06 20:09:47.032716 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.032700 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 20:09:47.046569 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.046545 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qvkzx"] Jul 06 20:09:47.122748 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.122726 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1601a025-e021-405e-9b41-1446d3a724c8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qvkzx\" (UID: \"1601a025-e021-405e-9b41-1446d3a724c8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qvkzx" Jul 06 20:09:47.122846 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.122767 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl62x\" (UniqueName: \"kubernetes.io/projected/1601a025-e021-405e-9b41-1446d3a724c8-kube-api-access-jl62x\") pod \"cert-manager-webhook-6486ffd997-qvkzx\" (UID: \"1601a025-e021-405e-9b41-1446d3a724c8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qvkzx" Jul 06 20:09:47.224054 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.224032 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1601a025-e021-405e-9b41-1446d3a724c8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qvkzx\" (UID: \"1601a025-e021-405e-9b41-1446d3a724c8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qvkzx" Jul 06 20:09:47.224138 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.224069 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl62x\" (UniqueName: \"kubernetes.io/projected/1601a025-e021-405e-9b41-1446d3a724c8-kube-api-access-jl62x\") pod \"cert-manager-webhook-6486ffd997-qvkzx\" (UID: \"1601a025-e021-405e-9b41-1446d3a724c8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qvkzx" Jul 06 20:09:47.256557 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.256539 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 20:09:47.256885 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.256851 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1601a025-e021-405e-9b41-1446d3a724c8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qvkzx\" (UID: \"1601a025-e021-405e-9b41-1446d3a724c8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qvkzx" Jul 06 20:09:47.267314 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.267296 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 20:09:47.277152 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.277109 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl62x\" (UniqueName: \"kubernetes.io/projected/1601a025-e021-405e-9b41-1446d3a724c8-kube-api-access-jl62x\") pod \"cert-manager-webhook-6486ffd997-qvkzx\" (UID: \"1601a025-e021-405e-9b41-1446d3a724c8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qvkzx" Jul 06 20:09:47.339094 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.339073 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-ws6wn\"" Jul 06 20:09:47.346924 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.346909 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qvkzx" Jul 06 20:09:47.449600 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.449573 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh"] Jul 06 20:09:47.452637 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.452617 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" Jul 06 20:09:47.456058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.455844 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nl5wp\"" Jul 06 20:09:47.456058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.455900 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 20:09:47.456058 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.455992 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 20:09:47.464818 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.464798 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh"] Jul 06 20:09:47.514167 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.512511 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qvkzx"] Jul 06 20:09:47.627151 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.627086 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4937861-3758-4e55-89c7-dcbbb94d5702-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh\" (UID: \"f4937861-3758-4e55-89c7-dcbbb94d5702\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" Jul 06 20:09:47.627250 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.627161 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhm8n\" (UniqueName: \"kubernetes.io/projected/f4937861-3758-4e55-89c7-dcbbb94d5702-kube-api-access-hhm8n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh\" (UID: \"f4937861-3758-4e55-89c7-dcbbb94d5702\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" Jul 06 20:09:47.627291 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.627267 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4937861-3758-4e55-89c7-dcbbb94d5702-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh\" (UID: \"f4937861-3758-4e55-89c7-dcbbb94d5702\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" Jul 06 20:09:47.714068 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.714043 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qvkzx" event={"ID":"1601a025-e021-405e-9b41-1446d3a724c8","Type":"ContainerStarted","Data":"95eac082cdfba9abf52ca0bb3847c194d1ea1928cc2d88f9fe0ab6510c18eb9e"} Jul 06 20:09:47.728447 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.728419 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhm8n\" (UniqueName: \"kubernetes.io/projected/f4937861-3758-4e55-89c7-dcbbb94d5702-kube-api-access-hhm8n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh\" (UID: \"f4937861-3758-4e55-89c7-dcbbb94d5702\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" Jul 06 20:09:47.728540 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.728465 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4937861-3758-4e55-89c7-dcbbb94d5702-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh\" (UID: \"f4937861-3758-4e55-89c7-dcbbb94d5702\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" Jul 06 20:09:47.728540 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.728495 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4937861-3758-4e55-89c7-dcbbb94d5702-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh\" (UID: \"f4937861-3758-4e55-89c7-dcbbb94d5702\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" Jul 06 20:09:47.728762 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.728743 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4937861-3758-4e55-89c7-dcbbb94d5702-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh\" (UID: \"f4937861-3758-4e55-89c7-dcbbb94d5702\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" Jul 06 20:09:47.728762 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.728767 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4937861-3758-4e55-89c7-dcbbb94d5702-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh\" (UID: \"f4937861-3758-4e55-89c7-dcbbb94d5702\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" Jul 06 20:09:47.743491 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.743461 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhm8n\" (UniqueName: \"kubernetes.io/projected/f4937861-3758-4e55-89c7-dcbbb94d5702-kube-api-access-hhm8n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh\" (UID: \"f4937861-3758-4e55-89c7-dcbbb94d5702\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" Jul 06 20:09:47.763651 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.763631 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" Jul 06 20:09:47.894703 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:47.894679 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh"] Jul 06 20:09:47.896535 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:09:47.896509 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4937861_3758_4e55_89c7_dcbbb94d5702.slice/crio-d2bd2dd8905c8f96f9702f03ba92c73fe3d2610ab7998c08288a8c5f584b66c9 WatchSource:0}: Error finding container d2bd2dd8905c8f96f9702f03ba92c73fe3d2610ab7998c08288a8c5f584b66c9: Status 404 returned error can't find the container with id d2bd2dd8905c8f96f9702f03ba92c73fe3d2610ab7998c08288a8c5f584b66c9 Jul 06 20:09:47.993304 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:09:47.993272 2583 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4937861_3758_4e55_89c7_dcbbb94d5702.slice/crio-b07838fa11da3c1e551cac616a669ff51ab20eae105c7a25813d741b5e7e1f8e.scope\": RecentStats: unable to find data in memory cache]" Jul 06 20:09:48.720418 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:48.720382 2583 generic.go:356] "Generic (PLEG): container finished" podID="f4937861-3758-4e55-89c7-dcbbb94d5702" containerID="b07838fa11da3c1e551cac616a669ff51ab20eae105c7a25813d741b5e7e1f8e" exitCode=0 Jul 06 20:09:48.720887 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:48.720475 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" event={"ID":"f4937861-3758-4e55-89c7-dcbbb94d5702","Type":"ContainerDied","Data":"b07838fa11da3c1e551cac616a669ff51ab20eae105c7a25813d741b5e7e1f8e"} Jul 06 20:09:48.720887 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:48.720536 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" event={"ID":"f4937861-3758-4e55-89c7-dcbbb94d5702","Type":"ContainerStarted","Data":"d2bd2dd8905c8f96f9702f03ba92c73fe3d2610ab7998c08288a8c5f584b66c9"} Jul 06 20:09:51.736721 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:51.736684 2583 generic.go:356] "Generic (PLEG): container finished" podID="f4937861-3758-4e55-89c7-dcbbb94d5702" containerID="7a88facd1743097c5912ae7c64a2190f6bca0b280cce6a4a77bf50e1c901989f" exitCode=0 Jul 06 20:09:51.737172 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:51.736774 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" event={"ID":"f4937861-3758-4e55-89c7-dcbbb94d5702","Type":"ContainerDied","Data":"7a88facd1743097c5912ae7c64a2190f6bca0b280cce6a4a77bf50e1c901989f"} Jul 06 20:09:51.738487 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:51.738464 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qvkzx" event={"ID":"1601a025-e021-405e-9b41-1446d3a724c8","Type":"ContainerStarted","Data":"e0b85628100b42d4d8f7e8c2e051810f8df943dd43a56c93c3d79ebb47452866"} Jul 06 20:09:51.738607 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:51.738589 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qvkzx" Jul 06 20:09:51.784525 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:51.784475 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-qvkzx" podStartSLOduration=2.61121653 podStartE2EDuration="5.784456421s" totalImagesPullingTime="3.173239891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:09:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:09:51.781556923 +0000 UTC m=+244.908469029" watchObservedRunningTime="2026-07-06 20:09:51.784456421 +0000 UTC m=+244.911368526" Jul 06 20:09:52.744847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:52.744812 2583 generic.go:356] "Generic (PLEG): container finished" podID="f4937861-3758-4e55-89c7-dcbbb94d5702" containerID="98113639a428123a31b36078ebe94eeae16ba438e84f17cb5d3f48fc728ac46e" exitCode=0 Jul 06 20:09:52.745303 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:52.744889 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" event={"ID":"f4937861-3758-4e55-89c7-dcbbb94d5702","Type":"ContainerDied","Data":"98113639a428123a31b36078ebe94eeae16ba438e84f17cb5d3f48fc728ac46e"} Jul 06 20:09:53.880931 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:53.880905 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" Jul 06 20:09:53.980591 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:53.980564 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4937861-3758-4e55-89c7-dcbbb94d5702-util\") pod \"f4937861-3758-4e55-89c7-dcbbb94d5702\" (UID: \"f4937861-3758-4e55-89c7-dcbbb94d5702\") " Jul 06 20:09:53.980713 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:53.980602 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhm8n\" (UniqueName: \"kubernetes.io/projected/f4937861-3758-4e55-89c7-dcbbb94d5702-kube-api-access-hhm8n\") pod \"f4937861-3758-4e55-89c7-dcbbb94d5702\" (UID: \"f4937861-3758-4e55-89c7-dcbbb94d5702\") " Jul 06 20:09:53.980713 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:53.980625 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4937861-3758-4e55-89c7-dcbbb94d5702-bundle\") pod \"f4937861-3758-4e55-89c7-dcbbb94d5702\" (UID: \"f4937861-3758-4e55-89c7-dcbbb94d5702\") " Jul 06 20:09:53.981051 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:53.981028 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4937861-3758-4e55-89c7-dcbbb94d5702-bundle" (OuterVolumeSpecName: "bundle") pod "f4937861-3758-4e55-89c7-dcbbb94d5702" (UID: "f4937861-3758-4e55-89c7-dcbbb94d5702"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:53.982697 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:53.982678 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4937861-3758-4e55-89c7-dcbbb94d5702-kube-api-access-hhm8n" (OuterVolumeSpecName: "kube-api-access-hhm8n") pod "f4937861-3758-4e55-89c7-dcbbb94d5702" (UID: "f4937861-3758-4e55-89c7-dcbbb94d5702"). InnerVolumeSpecName "kube-api-access-hhm8n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:09:53.986788 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:53.986765 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4937861-3758-4e55-89c7-dcbbb94d5702-util" (OuterVolumeSpecName: "util") pod "f4937861-3758-4e55-89c7-dcbbb94d5702" (UID: "f4937861-3758-4e55-89c7-dcbbb94d5702"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:09:54.081398 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:54.081332 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4937861-3758-4e55-89c7-dcbbb94d5702-util\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:09:54.081398 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:54.081356 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhm8n\" (UniqueName: \"kubernetes.io/projected/f4937861-3758-4e55-89c7-dcbbb94d5702-kube-api-access-hhm8n\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:09:54.081398 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:54.081372 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4937861-3758-4e55-89c7-dcbbb94d5702-bundle\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:09:54.754558 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:54.754531 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" Jul 06 20:09:54.754709 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:54.754534 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff66rh" event={"ID":"f4937861-3758-4e55-89c7-dcbbb94d5702","Type":"ContainerDied","Data":"d2bd2dd8905c8f96f9702f03ba92c73fe3d2610ab7998c08288a8c5f584b66c9"} Jul 06 20:09:54.754709 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:54.754649 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2bd2dd8905c8f96f9702f03ba92c73fe3d2610ab7998c08288a8c5f584b66c9" Jul 06 20:09:57.746929 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:09:57.746897 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qvkzx" Jul 06 20:10:04.425642 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.425605 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlztr"] Jul 06 20:10:04.426066 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.425955 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4937861-3758-4e55-89c7-dcbbb94d5702" containerName="pull" Jul 06 20:10:04.426066 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.425969 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4937861-3758-4e55-89c7-dcbbb94d5702" containerName="pull" Jul 06 20:10:04.426066 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.425979 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4937861-3758-4e55-89c7-dcbbb94d5702" containerName="util" Jul 06 20:10:04.426066 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.425984 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4937861-3758-4e55-89c7-dcbbb94d5702" containerName="util" Jul 06 20:10:04.426066 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.426014 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4937861-3758-4e55-89c7-dcbbb94d5702" containerName="extract" Jul 06 20:10:04.426066 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.426020 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4937861-3758-4e55-89c7-dcbbb94d5702" containerName="extract" Jul 06 20:10:04.426271 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.426077 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4937861-3758-4e55-89c7-dcbbb94d5702" containerName="extract" Jul 06 20:10:04.429650 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.429635 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlztr" Jul 06 20:10:04.432228 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.432205 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 20:10:04.433317 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.433296 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 20:10:04.433402 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.433318 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-k272z\"" Jul 06 20:10:04.444687 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.444666 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlztr"] Jul 06 20:10:04.460584 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.460561 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl4td\" (UniqueName: \"kubernetes.io/projected/87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd-kube-api-access-bl4td\") pod \"openshift-lws-operator-fd8ccff4c-hlztr\" (UID: \"87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlztr" Jul 06 20:10:04.460687 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.460599 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hlztr\" (UID: \"87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlztr" Jul 06 20:10:04.561006 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.560974 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl4td\" (UniqueName: \"kubernetes.io/projected/87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd-kube-api-access-bl4td\") pod \"openshift-lws-operator-fd8ccff4c-hlztr\" (UID: \"87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlztr" Jul 06 20:10:04.561190 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.561166 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hlztr\" (UID: \"87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlztr" Jul 06 20:10:04.561507 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.561484 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hlztr\" (UID: \"87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlztr" Jul 06 20:10:04.575416 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.575395 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl4td\" (UniqueName: \"kubernetes.io/projected/87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd-kube-api-access-bl4td\") pod \"openshift-lws-operator-fd8ccff4c-hlztr\" (UID: \"87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlztr" Jul 06 20:10:04.738746 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.738715 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlztr" Jul 06 20:10:04.871440 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:04.871413 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlztr"] Jul 06 20:10:04.873914 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:10:04.873883 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87e2e689_5dd8_4f87_ab4b_8a5d8f3096bd.slice/crio-269d47b8da5cd30cde64ef5df93da37e5fc06c940ed0fafb54814e95df34a077 WatchSource:0}: Error finding container 269d47b8da5cd30cde64ef5df93da37e5fc06c940ed0fafb54814e95df34a077: Status 404 returned error can't find the container with id 269d47b8da5cd30cde64ef5df93da37e5fc06c940ed0fafb54814e95df34a077 Jul 06 20:10:05.802746 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:05.802682 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlztr" event={"ID":"87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd","Type":"ContainerStarted","Data":"269d47b8da5cd30cde64ef5df93da37e5fc06c940ed0fafb54814e95df34a077"} Jul 06 20:10:06.809427 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:06.809391 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlztr" event={"ID":"87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd","Type":"ContainerStarted","Data":"cebe56a5715469d331c4684d2e3fc69d6b6fcad45b8ea19360c9fce8e275a0a9"} Jul 06 20:10:06.837369 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:06.837316 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hlztr" podStartSLOduration=1.360585703 podStartE2EDuration="2.837297444s" totalImagesPullingTime="1.476711741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:10:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:10:06.835855736 +0000 UTC m=+259.962767839" watchObservedRunningTime="2026-07-06 20:10:06.837297444 +0000 UTC m=+259.964209551" Jul 06 20:10:09.831272 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:09.831195 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn"] Jul 06 20:10:09.848354 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:09.848326 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn"] Jul 06 20:10:09.848493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:09.848425 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" Jul 06 20:10:09.851385 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:09.851366 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 20:10:09.851587 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:09.851569 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 20:10:09.852355 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:09.852341 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nl5wp\"" Jul 06 20:10:09.904317 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:09.904296 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/827a50b9-17ae-4320-8a41-5c184d5e95d8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn\" (UID: \"827a50b9-17ae-4320-8a41-5c184d5e95d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" Jul 06 20:10:09.904413 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:09.904364 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6sjh\" (UniqueName: \"kubernetes.io/projected/827a50b9-17ae-4320-8a41-5c184d5e95d8-kube-api-access-w6sjh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn\" (UID: \"827a50b9-17ae-4320-8a41-5c184d5e95d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" Jul 06 20:10:09.904413 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:09.904405 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/827a50b9-17ae-4320-8a41-5c184d5e95d8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn\" (UID: \"827a50b9-17ae-4320-8a41-5c184d5e95d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" Jul 06 20:10:10.005039 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:10.005015 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6sjh\" (UniqueName: \"kubernetes.io/projected/827a50b9-17ae-4320-8a41-5c184d5e95d8-kube-api-access-w6sjh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn\" (UID: \"827a50b9-17ae-4320-8a41-5c184d5e95d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" Jul 06 20:10:10.005128 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:10.005056 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/827a50b9-17ae-4320-8a41-5c184d5e95d8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn\" (UID: \"827a50b9-17ae-4320-8a41-5c184d5e95d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" Jul 06 20:10:10.005128 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:10.005084 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/827a50b9-17ae-4320-8a41-5c184d5e95d8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn\" (UID: \"827a50b9-17ae-4320-8a41-5c184d5e95d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" Jul 06 20:10:10.005376 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:10.005357 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/827a50b9-17ae-4320-8a41-5c184d5e95d8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn\" (UID: \"827a50b9-17ae-4320-8a41-5c184d5e95d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" Jul 06 20:10:10.005462 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:10.005447 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/827a50b9-17ae-4320-8a41-5c184d5e95d8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn\" (UID: \"827a50b9-17ae-4320-8a41-5c184d5e95d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" Jul 06 20:10:10.019163 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:10.019146 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6sjh\" (UniqueName: \"kubernetes.io/projected/827a50b9-17ae-4320-8a41-5c184d5e95d8-kube-api-access-w6sjh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn\" (UID: \"827a50b9-17ae-4320-8a41-5c184d5e95d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" Jul 06 20:10:10.157914 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:10.157850 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" Jul 06 20:10:10.309790 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:10.309768 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn"] Jul 06 20:10:10.311934 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:10:10.311907 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod827a50b9_17ae_4320_8a41_5c184d5e95d8.slice/crio-85553ac08841c3737bd4e1bfdf23f88f1f253bf36f60a5445a536432f77c3e69 WatchSource:0}: Error finding container 85553ac08841c3737bd4e1bfdf23f88f1f253bf36f60a5445a536432f77c3e69: Status 404 returned error can't find the container with id 85553ac08841c3737bd4e1bfdf23f88f1f253bf36f60a5445a536432f77c3e69 Jul 06 20:10:10.829137 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:10.829099 2583 generic.go:356] "Generic (PLEG): container finished" podID="827a50b9-17ae-4320-8a41-5c184d5e95d8" containerID="9f7f53925f18d586cdcfd58188624927fe4011969284002e4da7a827674f1107" exitCode=0 Jul 06 20:10:10.829321 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:10.829190 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" event={"ID":"827a50b9-17ae-4320-8a41-5c184d5e95d8","Type":"ContainerDied","Data":"9f7f53925f18d586cdcfd58188624927fe4011969284002e4da7a827674f1107"} Jul 06 20:10:10.829321 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:10.829250 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" event={"ID":"827a50b9-17ae-4320-8a41-5c184d5e95d8","Type":"ContainerStarted","Data":"85553ac08841c3737bd4e1bfdf23f88f1f253bf36f60a5445a536432f77c3e69"} Jul 06 20:10:12.842647 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:12.842610 2583 generic.go:356] "Generic (PLEG): container finished" podID="827a50b9-17ae-4320-8a41-5c184d5e95d8" containerID="6ed5aa41e15aa6938e75103bec39bb12f64fb3bac5d169ddf0abd6f36ac7bf5e" exitCode=0 Jul 06 20:10:12.843054 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:12.842669 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" event={"ID":"827a50b9-17ae-4320-8a41-5c184d5e95d8","Type":"ContainerDied","Data":"6ed5aa41e15aa6938e75103bec39bb12f64fb3bac5d169ddf0abd6f36ac7bf5e"} Jul 06 20:10:13.849238 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:13.849200 2583 generic.go:356] "Generic (PLEG): container finished" podID="827a50b9-17ae-4320-8a41-5c184d5e95d8" containerID="c92d9b19b2e82b17ca6b8aa62aa2f1887e9107097b4863e95b1fea3935bf1e51" exitCode=0 Jul 06 20:10:13.849692 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:13.849275 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" event={"ID":"827a50b9-17ae-4320-8a41-5c184d5e95d8","Type":"ContainerDied","Data":"c92d9b19b2e82b17ca6b8aa62aa2f1887e9107097b4863e95b1fea3935bf1e51"} Jul 06 20:10:14.978124 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:14.978076 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" Jul 06 20:10:15.048307 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:15.048280 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/827a50b9-17ae-4320-8a41-5c184d5e95d8-util\") pod \"827a50b9-17ae-4320-8a41-5c184d5e95d8\" (UID: \"827a50b9-17ae-4320-8a41-5c184d5e95d8\") " Jul 06 20:10:15.048442 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:15.048327 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/827a50b9-17ae-4320-8a41-5c184d5e95d8-bundle\") pod \"827a50b9-17ae-4320-8a41-5c184d5e95d8\" (UID: \"827a50b9-17ae-4320-8a41-5c184d5e95d8\") " Jul 06 20:10:15.048442 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:15.048396 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6sjh\" (UniqueName: \"kubernetes.io/projected/827a50b9-17ae-4320-8a41-5c184d5e95d8-kube-api-access-w6sjh\") pod \"827a50b9-17ae-4320-8a41-5c184d5e95d8\" (UID: \"827a50b9-17ae-4320-8a41-5c184d5e95d8\") " Jul 06 20:10:15.049082 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:15.049047 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/827a50b9-17ae-4320-8a41-5c184d5e95d8-bundle" (OuterVolumeSpecName: "bundle") pod "827a50b9-17ae-4320-8a41-5c184d5e95d8" (UID: "827a50b9-17ae-4320-8a41-5c184d5e95d8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:10:15.050481 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:15.050455 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/827a50b9-17ae-4320-8a41-5c184d5e95d8-kube-api-access-w6sjh" (OuterVolumeSpecName: "kube-api-access-w6sjh") pod "827a50b9-17ae-4320-8a41-5c184d5e95d8" (UID: "827a50b9-17ae-4320-8a41-5c184d5e95d8"). InnerVolumeSpecName "kube-api-access-w6sjh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:10:15.054126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:15.054107 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/827a50b9-17ae-4320-8a41-5c184d5e95d8-util" (OuterVolumeSpecName: "util") pod "827a50b9-17ae-4320-8a41-5c184d5e95d8" (UID: "827a50b9-17ae-4320-8a41-5c184d5e95d8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:10:15.149325 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:15.149270 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/827a50b9-17ae-4320-8a41-5c184d5e95d8-bundle\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:10:15.149325 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:15.149291 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6sjh\" (UniqueName: \"kubernetes.io/projected/827a50b9-17ae-4320-8a41-5c184d5e95d8-kube-api-access-w6sjh\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:10:15.149325 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:15.149302 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/827a50b9-17ae-4320-8a41-5c184d5e95d8-util\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:10:15.859688 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:15.859624 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" Jul 06 20:10:15.859688 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:15.859641 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aflmrmn" event={"ID":"827a50b9-17ae-4320-8a41-5c184d5e95d8","Type":"ContainerDied","Data":"85553ac08841c3737bd4e1bfdf23f88f1f253bf36f60a5445a536432f77c3e69"} Jul 06 20:10:15.859688 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:15.859684 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85553ac08841c3737bd4e1bfdf23f88f1f253bf36f60a5445a536432f77c3e69" Jul 06 20:10:27.815261 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.815228 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm"] Jul 06 20:10:27.815673 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.815579 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="827a50b9-17ae-4320-8a41-5c184d5e95d8" containerName="util" Jul 06 20:10:27.815673 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.815591 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="827a50b9-17ae-4320-8a41-5c184d5e95d8" containerName="util" Jul 06 20:10:27.815673 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.815613 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="827a50b9-17ae-4320-8a41-5c184d5e95d8" containerName="pull" Jul 06 20:10:27.815673 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.815618 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="827a50b9-17ae-4320-8a41-5c184d5e95d8" containerName="pull" Jul 06 20:10:27.815673 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.815624 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="827a50b9-17ae-4320-8a41-5c184d5e95d8" containerName="extract" Jul 06 20:10:27.815673 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.815629 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="827a50b9-17ae-4320-8a41-5c184d5e95d8" containerName="extract" Jul 06 20:10:27.815860 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.815683 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="827a50b9-17ae-4320-8a41-5c184d5e95d8" containerName="extract" Jul 06 20:10:27.818950 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.818933 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:27.821552 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.821529 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 20:10:27.822004 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.821989 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 20:10:27.822094 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.822035 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-crnc6\"" Jul 06 20:10:27.822157 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.822114 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 20:10:27.822209 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.822195 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 20:10:27.835489 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.835467 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm"] Jul 06 20:10:27.946310 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.946286 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5f8a799-9d7a-498c-8e6d-8baa3036efab-tmp\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:27.946448 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.946326 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a5f8a799-9d7a-498c-8e6d-8baa3036efab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:27.946448 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.946372 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwhsk\" (UniqueName: \"kubernetes.io/projected/a5f8a799-9d7a-498c-8e6d-8baa3036efab-kube-api-access-lwhsk\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:27.946448 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.946404 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a5f8a799-9d7a-498c-8e6d-8baa3036efab-manifests\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:27.946448 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:27.946428 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a5f8a799-9d7a-498c-8e6d-8baa3036efab-webhook-cert\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:28.046921 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:28.046894 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwhsk\" (UniqueName: \"kubernetes.io/projected/a5f8a799-9d7a-498c-8e6d-8baa3036efab-kube-api-access-lwhsk\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:28.047026 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:28.046928 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a5f8a799-9d7a-498c-8e6d-8baa3036efab-manifests\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:28.047026 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:28.046957 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a5f8a799-9d7a-498c-8e6d-8baa3036efab-webhook-cert\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:28.047026 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:28.046987 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5f8a799-9d7a-498c-8e6d-8baa3036efab-tmp\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:28.047026 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:28.047018 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a5f8a799-9d7a-498c-8e6d-8baa3036efab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:28.047334 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:28.047308 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a5f8a799-9d7a-498c-8e6d-8baa3036efab-manifests\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:28.049205 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:28.049185 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5f8a799-9d7a-498c-8e6d-8baa3036efab-tmp\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:28.049494 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:28.049478 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a5f8a799-9d7a-498c-8e6d-8baa3036efab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:28.049627 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:28.049608 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a5f8a799-9d7a-498c-8e6d-8baa3036efab-webhook-cert\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:28.061063 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:28.061040 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwhsk\" (UniqueName: \"kubernetes.io/projected/a5f8a799-9d7a-498c-8e6d-8baa3036efab-kube-api-access-lwhsk\") pod \"opendatahub-operator-controller-manager-68cbf59454-jvfwm\" (UID: \"a5f8a799-9d7a-498c-8e6d-8baa3036efab\") " pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:28.128966 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:28.128908 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:28.277400 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:28.277373 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm"] Jul 06 20:10:28.279009 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:10:28.278984 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f8a799_9d7a_498c_8e6d_8baa3036efab.slice/crio-569805dbc19cad6af7acd73ae87ae58cebdf3927ea5c38cde4db47b749ebaf07 WatchSource:0}: Error finding container 569805dbc19cad6af7acd73ae87ae58cebdf3927ea5c38cde4db47b749ebaf07: Status 404 returned error can't find the container with id 569805dbc19cad6af7acd73ae87ae58cebdf3927ea5c38cde4db47b749ebaf07 Jul 06 20:10:28.918299 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:28.918267 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" event={"ID":"a5f8a799-9d7a-498c-8e6d-8baa3036efab","Type":"ContainerStarted","Data":"569805dbc19cad6af7acd73ae87ae58cebdf3927ea5c38cde4db47b749ebaf07"} Jul 06 20:10:31.935926 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:31.935840 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" event={"ID":"a5f8a799-9d7a-498c-8e6d-8baa3036efab","Type":"ContainerStarted","Data":"e0f9e5746994b7ba705af43a85e7203792eee11c3a53265fda75accfe6a6f77a"} Jul 06 20:10:32.941911 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:32.941824 2583 generic.go:356] "Generic (PLEG): container finished" podID="a5f8a799-9d7a-498c-8e6d-8baa3036efab" containerID="e0f9e5746994b7ba705af43a85e7203792eee11c3a53265fda75accfe6a6f77a" exitCode=0 Jul 06 20:10:32.942237 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:32.941908 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" event={"ID":"a5f8a799-9d7a-498c-8e6d-8baa3036efab","Type":"ContainerDied","Data":"e0f9e5746994b7ba705af43a85e7203792eee11c3a53265fda75accfe6a6f77a"} Jul 06 20:10:33.950514 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:33.950478 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" event={"ID":"a5f8a799-9d7a-498c-8e6d-8baa3036efab","Type":"ContainerStarted","Data":"be1c990f5f454f355177d1a746a3205e3204490d2d9570bb4afb6a94672bc5d4"} Jul 06 20:10:33.951013 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:33.950619 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:33.982013 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:33.981964 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" podStartSLOduration=2.317343985 podStartE2EDuration="6.981949414s" totalImagesPullingTime="3.585098041s" totalInitContainerRuntime="1.079507388s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:10:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:10:33.978906291 +0000 UTC m=+287.105818399" watchObservedRunningTime="2026-07-06 20:10:33.981949414 +0000 UTC m=+287.108861518" Jul 06 20:10:44.958603 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:44.958570 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:10:47.258496 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:47.258467 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:10:47.268691 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:47.268663 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:10:47.271086 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:47.271067 2583 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 20:10:47.733787 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:47.733748 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb"] Jul 06 20:10:47.736827 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:47.736811 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb" Jul 06 20:10:47.739253 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:47.739227 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 20:10:47.739542 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:47.739528 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tsrxb\"" Jul 06 20:10:47.739717 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:47.739700 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 20:10:47.771093 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:47.771068 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb"] Jul 06 20:10:47.903661 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:47.903627 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvvq4\" (UniqueName: \"kubernetes.io/projected/7678ddd9-44b2-49b0-9bc2-da640345aea2-kube-api-access-hvvq4\") pod \"kube-auth-proxy-64f5c756bf-p8brb\" (UID: \"7678ddd9-44b2-49b0-9bc2-da640345aea2\") " pod="openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb" Jul 06 20:10:47.903819 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:47.903667 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7678ddd9-44b2-49b0-9bc2-da640345aea2-tmp\") pod \"kube-auth-proxy-64f5c756bf-p8brb\" (UID: \"7678ddd9-44b2-49b0-9bc2-da640345aea2\") " pod="openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb" Jul 06 20:10:47.903819 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:47.903718 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7678ddd9-44b2-49b0-9bc2-da640345aea2-tls-certs\") pod \"kube-auth-proxy-64f5c756bf-p8brb\" (UID: \"7678ddd9-44b2-49b0-9bc2-da640345aea2\") " pod="openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb" Jul 06 20:10:48.004467 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.004381 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7678ddd9-44b2-49b0-9bc2-da640345aea2-tls-certs\") pod \"kube-auth-proxy-64f5c756bf-p8brb\" (UID: \"7678ddd9-44b2-49b0-9bc2-da640345aea2\") " pod="openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb" Jul 06 20:10:48.004467 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.004466 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvvq4\" (UniqueName: \"kubernetes.io/projected/7678ddd9-44b2-49b0-9bc2-da640345aea2-kube-api-access-hvvq4\") pod \"kube-auth-proxy-64f5c756bf-p8brb\" (UID: \"7678ddd9-44b2-49b0-9bc2-da640345aea2\") " pod="openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb" Jul 06 20:10:48.004656 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.004494 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7678ddd9-44b2-49b0-9bc2-da640345aea2-tmp\") pod \"kube-auth-proxy-64f5c756bf-p8brb\" (UID: \"7678ddd9-44b2-49b0-9bc2-da640345aea2\") " pod="openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb" Jul 06 20:10:48.006611 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.006586 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7678ddd9-44b2-49b0-9bc2-da640345aea2-tmp\") pod \"kube-auth-proxy-64f5c756bf-p8brb\" (UID: \"7678ddd9-44b2-49b0-9bc2-da640345aea2\") " pod="openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb" Jul 06 20:10:48.006836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.006816 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7678ddd9-44b2-49b0-9bc2-da640345aea2-tls-certs\") pod \"kube-auth-proxy-64f5c756bf-p8brb\" (UID: \"7678ddd9-44b2-49b0-9bc2-da640345aea2\") " pod="openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb" Jul 06 20:10:48.031020 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.030983 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvvq4\" (UniqueName: \"kubernetes.io/projected/7678ddd9-44b2-49b0-9bc2-da640345aea2-kube-api-access-hvvq4\") pod \"kube-auth-proxy-64f5c756bf-p8brb\" (UID: \"7678ddd9-44b2-49b0-9bc2-da640345aea2\") " pod="openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb" Jul 06 20:10:48.046510 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.046487 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb" Jul 06 20:10:48.140411 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.140375 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-np686"] Jul 06 20:10:48.144802 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.144779 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-np686" Jul 06 20:10:48.147672 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.147606 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 06 20:10:48.147967 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.147943 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-wqzlk\"" Jul 06 20:10:48.163033 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.162962 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-np686"] Jul 06 20:10:48.214582 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.214562 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb"] Jul 06 20:10:48.216374 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:10:48.216348 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7678ddd9_44b2_49b0_9bc2_da640345aea2.slice/crio-37491646485ec461504fbecda6e42732fc0563121c0eb7666583df0f30bc4ddc WatchSource:0}: Error finding container 37491646485ec461504fbecda6e42732fc0563121c0eb7666583df0f30bc4ddc: Status 404 returned error can't find the container with id 37491646485ec461504fbecda6e42732fc0563121c0eb7666583df0f30bc4ddc Jul 06 20:10:48.307126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.307054 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jxcw\" (UniqueName: \"kubernetes.io/projected/ed7b2a0f-049e-4874-8a09-076638035822-kube-api-access-6jxcw\") pod \"odh-model-controller-7998c67dc4-np686\" (UID: \"ed7b2a0f-049e-4874-8a09-076638035822\") " pod="opendatahub/odh-model-controller-7998c67dc4-np686" Jul 06 20:10:48.307126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.307084 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed7b2a0f-049e-4874-8a09-076638035822-cert\") pod \"odh-model-controller-7998c67dc4-np686\" (UID: \"ed7b2a0f-049e-4874-8a09-076638035822\") " pod="opendatahub/odh-model-controller-7998c67dc4-np686" Jul 06 20:10:48.407947 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.407865 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jxcw\" (UniqueName: \"kubernetes.io/projected/ed7b2a0f-049e-4874-8a09-076638035822-kube-api-access-6jxcw\") pod \"odh-model-controller-7998c67dc4-np686\" (UID: \"ed7b2a0f-049e-4874-8a09-076638035822\") " pod="opendatahub/odh-model-controller-7998c67dc4-np686" Jul 06 20:10:48.407947 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.407943 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed7b2a0f-049e-4874-8a09-076638035822-cert\") pod \"odh-model-controller-7998c67dc4-np686\" (UID: \"ed7b2a0f-049e-4874-8a09-076638035822\") " pod="opendatahub/odh-model-controller-7998c67dc4-np686" Jul 06 20:10:48.408189 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:10:48.408139 2583 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 06 20:10:48.408248 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:10:48.408197 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed7b2a0f-049e-4874-8a09-076638035822-cert podName:ed7b2a0f-049e-4874-8a09-076638035822 nodeName:}" failed. No retries permitted until 2026-07-06 20:10:48.908177589 +0000 UTC m=+302.035089684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ed7b2a0f-049e-4874-8a09-076638035822-cert") pod "odh-model-controller-7998c67dc4-np686" (UID: "ed7b2a0f-049e-4874-8a09-076638035822") : secret "odh-model-controller-webhook-cert" not found Jul 06 20:10:48.429173 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.429142 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jxcw\" (UniqueName: \"kubernetes.io/projected/ed7b2a0f-049e-4874-8a09-076638035822-kube-api-access-6jxcw\") pod \"odh-model-controller-7998c67dc4-np686\" (UID: \"ed7b2a0f-049e-4874-8a09-076638035822\") " pod="opendatahub/odh-model-controller-7998c67dc4-np686" Jul 06 20:10:48.912855 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.912824 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed7b2a0f-049e-4874-8a09-076638035822-cert\") pod \"odh-model-controller-7998c67dc4-np686\" (UID: \"ed7b2a0f-049e-4874-8a09-076638035822\") " pod="opendatahub/odh-model-controller-7998c67dc4-np686" Jul 06 20:10:48.915132 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:48.915113 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed7b2a0f-049e-4874-8a09-076638035822-cert\") pod \"odh-model-controller-7998c67dc4-np686\" (UID: \"ed7b2a0f-049e-4874-8a09-076638035822\") " pod="opendatahub/odh-model-controller-7998c67dc4-np686" Jul 06 20:10:49.023266 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.023223 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb" event={"ID":"7678ddd9-44b2-49b0-9bc2-da640345aea2","Type":"ContainerStarted","Data":"37491646485ec461504fbecda6e42732fc0563121c0eb7666583df0f30bc4ddc"} Jul 06 20:10:49.060892 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.060566 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-np686" Jul 06 20:10:49.224587 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.224503 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb"] Jul 06 20:10:49.228583 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.228554 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.229105 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.229083 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-np686"] Jul 06 20:10:49.232387 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.232360 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 20:10:49.232596 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.232577 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 20:10:49.232991 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.232745 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fbzlz\"" Jul 06 20:10:49.232991 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.232803 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 20:10:49.233153 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:10:49.233034 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7b2a0f_049e_4874_8a09_076638035822.slice/crio-ba66f86c3a6c7264a8cac3078dc998213c152df0d52abe0d2767c17d9e90d13a WatchSource:0}: Error finding container ba66f86c3a6c7264a8cac3078dc998213c152df0d52abe0d2767c17d9e90d13a: Status 404 returned error can't find the container with id ba66f86c3a6c7264a8cac3078dc998213c152df0d52abe0d2767c17d9e90d13a Jul 06 20:10:49.233153 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.233050 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 20:10:49.234860 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.234837 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:10:49.246470 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.246444 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb"] Jul 06 20:10:49.315999 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.315971 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-cacerts\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.316392 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.316020 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.316392 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.316068 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhpct\" (UniqueName: \"kubernetes.io/projected/173ee089-1baa-4630-8a05-b994cfde37c5-kube-api-access-dhpct\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.316392 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.316085 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/173ee089-1baa-4630-8a05-b994cfde37c5-local-certs\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.316392 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.316148 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173ee089-1baa-4630-8a05-b994cfde37c5-istio-token\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.316392 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.316239 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.316392 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.316289 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/173ee089-1baa-4630-8a05-b994cfde37c5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.417566 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.417499 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.417566 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.417562 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhpct\" (UniqueName: \"kubernetes.io/projected/173ee089-1baa-4630-8a05-b994cfde37c5-kube-api-access-dhpct\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.417823 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.417591 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/173ee089-1baa-4630-8a05-b994cfde37c5-local-certs\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.417823 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.417628 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173ee089-1baa-4630-8a05-b994cfde37c5-istio-token\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.417823 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.417683 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.417823 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.417719 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/173ee089-1baa-4630-8a05-b994cfde37c5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.417823 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.417771 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-cacerts\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.418833 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.418783 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/173ee089-1baa-4630-8a05-b994cfde37c5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.420714 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.420689 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/173ee089-1baa-4630-8a05-b994cfde37c5-local-certs\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.420714 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.420706 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.420909 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.420867 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.420909 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.420897 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-cacerts\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.432319 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.432243 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173ee089-1baa-4630-8a05-b994cfde37c5-istio-token\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.433307 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.433282 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhpct\" (UniqueName: \"kubernetes.io/projected/173ee089-1baa-4630-8a05-b994cfde37c5-kube-api-access-dhpct\") pod \"istiod-openshift-gateway-7574459656-lmjnb\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.552671 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.552642 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:49.701983 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:49.701959 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb"] Jul 06 20:10:49.703795 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:10:49.703768 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod173ee089_1baa_4630_8a05_b994cfde37c5.slice/crio-6a24d802ed22300b76d3bd89e89c679becb35c9acc0f054d4caa5ebc339e6b4b WatchSource:0}: Error finding container 6a24d802ed22300b76d3bd89e89c679becb35c9acc0f054d4caa5ebc339e6b4b: Status 404 returned error can't find the container with id 6a24d802ed22300b76d3bd89e89c679becb35c9acc0f054d4caa5ebc339e6b4b Jul 06 20:10:50.032829 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:50.032789 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" event={"ID":"173ee089-1baa-4630-8a05-b994cfde37c5","Type":"ContainerStarted","Data":"6a24d802ed22300b76d3bd89e89c679becb35c9acc0f054d4caa5ebc339e6b4b"} Jul 06 20:10:50.034735 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:50.034695 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-np686" event={"ID":"ed7b2a0f-049e-4874-8a09-076638035822","Type":"ContainerStarted","Data":"ba66f86c3a6c7264a8cac3078dc998213c152df0d52abe0d2767c17d9e90d13a"} Jul 06 20:10:53.990753 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:53.990719 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:10:53.991070 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:53.990783 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:10:55.067628 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:55.067591 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-np686" event={"ID":"ed7b2a0f-049e-4874-8a09-076638035822","Type":"ContainerStarted","Data":"23d0afd561fcd9931c3b983083f8bb179d1169e347d4de2cf68c5c4e93856a4b"} Jul 06 20:10:55.068117 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:55.067695 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-np686" Jul 06 20:10:55.069370 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:55.069346 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" event={"ID":"173ee089-1baa-4630-8a05-b994cfde37c5","Type":"ContainerStarted","Data":"60b08ff115478f4eab572e9a68e6faf5cb5cb3f33dcc0a853d3321ae4e359e21"} Jul 06 20:10:55.069513 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:55.069482 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:10:55.070995 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:55.070969 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb" event={"ID":"7678ddd9-44b2-49b0-9bc2-da640345aea2","Type":"ContainerStarted","Data":"920c4e94920867cfdef89256be51cc72398707c593d6e1df0a4827302fbac327"} Jul 06 20:10:55.071516 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:55.071490 2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-lmjnb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 20:10:55.071609 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:55.071542 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" podUID="173ee089-1baa-4630-8a05-b994cfde37c5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 20:10:55.087965 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:55.087921 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-np686" podStartSLOduration=2.323978714 podStartE2EDuration="7.087904361s" totalImagesPullingTime="4.763925647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:10:55.086557548 +0000 UTC m=+308.213469653" watchObservedRunningTime="2026-07-06 20:10:55.087904361 +0000 UTC m=+308.214816463" Jul 06 20:10:55.109256 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:55.109210 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-64f5c756bf-p8brb" podStartSLOduration=2.339575962 podStartE2EDuration="8.109194628s" totalImagesPullingTime="5.769618666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:10:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:10:55.109012623 +0000 UTC m=+308.235924743" watchObservedRunningTime="2026-07-06 20:10:55.109194628 +0000 UTC m=+308.236106734" Jul 06 20:10:55.140647 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:55.140595 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" podStartSLOduration=1.85590156 podStartE2EDuration="6.140580451s" totalImagesPullingTime="4.284678891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:10:55.138640874 +0000 UTC m=+308.265552981" watchObservedRunningTime="2026-07-06 20:10:55.140580451 +0000 UTC m=+308.267492555" Jul 06 20:10:56.076739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:10:56.076712 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:11:00.027536 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.027503 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr"] Jul 06 20:11:00.031597 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.031575 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" Jul 06 20:11:00.037556 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.037536 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-8fvmv\"" Jul 06 20:11:00.037651 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.037535 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 20:11:00.045697 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.045676 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bfvds"] Jul 06 20:11:00.049071 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.049053 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" Jul 06 20:11:00.051736 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.051719 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jcpct\"" Jul 06 20:11:00.052491 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.052307 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 06 20:11:00.054912 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.054886 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr"] Jul 06 20:11:00.065409 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.065386 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bfvds"] Jul 06 20:11:00.107552 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.107515 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4c49034e-79c9-407b-b068-96bbe8d61dac-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7vpbr\" (UID: \"4c49034e-79c9-407b-b068-96bbe8d61dac\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" Jul 06 20:11:00.107678 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.107579 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm84w\" (UniqueName: \"kubernetes.io/projected/73340a04-d613-40d5-85ab-7c54daeaa757-kube-api-access-bm84w\") pod \"kserve-controller-manager-d6f4b65b8-bfvds\" (UID: \"73340a04-d613-40d5-85ab-7c54daeaa757\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" Jul 06 20:11:00.107678 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.107646 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5cj2\" (UniqueName: \"kubernetes.io/projected/4c49034e-79c9-407b-b068-96bbe8d61dac-kube-api-access-v5cj2\") pod \"llmisvc-controller-manager-777c6bd7d5-7vpbr\" (UID: \"4c49034e-79c9-407b-b068-96bbe8d61dac\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" Jul 06 20:11:00.107761 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.107684 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73340a04-d613-40d5-85ab-7c54daeaa757-cert\") pod \"kserve-controller-manager-d6f4b65b8-bfvds\" (UID: \"73340a04-d613-40d5-85ab-7c54daeaa757\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" Jul 06 20:11:00.208519 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.208490 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5cj2\" (UniqueName: \"kubernetes.io/projected/4c49034e-79c9-407b-b068-96bbe8d61dac-kube-api-access-v5cj2\") pod \"llmisvc-controller-manager-777c6bd7d5-7vpbr\" (UID: \"4c49034e-79c9-407b-b068-96bbe8d61dac\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" Jul 06 20:11:00.208625 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.208530 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73340a04-d613-40d5-85ab-7c54daeaa757-cert\") pod \"kserve-controller-manager-d6f4b65b8-bfvds\" (UID: \"73340a04-d613-40d5-85ab-7c54daeaa757\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" Jul 06 20:11:00.208625 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.208551 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4c49034e-79c9-407b-b068-96bbe8d61dac-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7vpbr\" (UID: \"4c49034e-79c9-407b-b068-96bbe8d61dac\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" Jul 06 20:11:00.208726 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.208624 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm84w\" (UniqueName: \"kubernetes.io/projected/73340a04-d613-40d5-85ab-7c54daeaa757-kube-api-access-bm84w\") pod \"kserve-controller-manager-d6f4b65b8-bfvds\" (UID: \"73340a04-d613-40d5-85ab-7c54daeaa757\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" Jul 06 20:11:00.211243 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.211220 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4c49034e-79c9-407b-b068-96bbe8d61dac-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7vpbr\" (UID: \"4c49034e-79c9-407b-b068-96bbe8d61dac\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" Jul 06 20:11:00.211318 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.211220 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73340a04-d613-40d5-85ab-7c54daeaa757-cert\") pod \"kserve-controller-manager-d6f4b65b8-bfvds\" (UID: \"73340a04-d613-40d5-85ab-7c54daeaa757\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" Jul 06 20:11:00.219257 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.219228 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5cj2\" (UniqueName: \"kubernetes.io/projected/4c49034e-79c9-407b-b068-96bbe8d61dac-kube-api-access-v5cj2\") pod \"llmisvc-controller-manager-777c6bd7d5-7vpbr\" (UID: \"4c49034e-79c9-407b-b068-96bbe8d61dac\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" Jul 06 20:11:00.219363 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.219350 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm84w\" (UniqueName: \"kubernetes.io/projected/73340a04-d613-40d5-85ab-7c54daeaa757-kube-api-access-bm84w\") pod \"kserve-controller-manager-d6f4b65b8-bfvds\" (UID: \"73340a04-d613-40d5-85ab-7c54daeaa757\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" Jul 06 20:11:00.341144 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.341065 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" Jul 06 20:11:00.360054 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.359741 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" Jul 06 20:11:00.551400 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.551360 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr"] Jul 06 20:11:00.552926 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:11:00.552896 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4c49034e_79c9_407b_b068_96bbe8d61dac.slice/crio-ee2411159b3157c03e5d2f92803e9e86ecd9b8590717d2f4087371b67ffdb41e WatchSource:0}: Error finding container ee2411159b3157c03e5d2f92803e9e86ecd9b8590717d2f4087371b67ffdb41e: Status 404 returned error can't find the container with id ee2411159b3157c03e5d2f92803e9e86ecd9b8590717d2f4087371b67ffdb41e Jul 06 20:11:00.573823 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:00.573783 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bfvds"] Jul 06 20:11:00.576850 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:11:00.576817 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73340a04_d613_40d5_85ab_7c54daeaa757.slice/crio-ee73b103072099df63cfe1e3863cb53a3bed651884f43d14b936cf7bc4597f86 WatchSource:0}: Error finding container ee73b103072099df63cfe1e3863cb53a3bed651884f43d14b936cf7bc4597f86: Status 404 returned error can't find the container with id ee73b103072099df63cfe1e3863cb53a3bed651884f43d14b936cf7bc4597f86 Jul 06 20:11:01.108030 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:01.107994 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" event={"ID":"73340a04-d613-40d5-85ab-7c54daeaa757","Type":"ContainerStarted","Data":"ee73b103072099df63cfe1e3863cb53a3bed651884f43d14b936cf7bc4597f86"} Jul 06 20:11:01.109290 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:01.109267 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" event={"ID":"4c49034e-79c9-407b-b068-96bbe8d61dac","Type":"ContainerStarted","Data":"ee2411159b3157c03e5d2f92803e9e86ecd9b8590717d2f4087371b67ffdb41e"} Jul 06 20:11:03.554413 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:03.554372 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb"] Jul 06 20:11:03.554864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:03.554573 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" podUID="173ee089-1baa-4630-8a05-b994cfde37c5" containerName="discovery" containerID="cri-o://60b08ff115478f4eab572e9a68e6faf5cb5cb3f33dcc0a853d3321ae4e359e21" gracePeriod=30 Jul 06 20:11:04.126684 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.126647 2583 generic.go:356] "Generic (PLEG): container finished" podID="173ee089-1baa-4630-8a05-b994cfde37c5" containerID="60b08ff115478f4eab572e9a68e6faf5cb5cb3f33dcc0a853d3321ae4e359e21" exitCode=0 Jul 06 20:11:04.126853 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.126697 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" event={"ID":"173ee089-1baa-4630-8a05-b994cfde37c5","Type":"ContainerDied","Data":"60b08ff115478f4eab572e9a68e6faf5cb5cb3f33dcc0a853d3321ae4e359e21"} Jul 06 20:11:04.329866 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.329837 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:11:04.452363 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.452054 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-istio-kubeconfig\") pod \"173ee089-1baa-4630-8a05-b994cfde37c5\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " Jul 06 20:11:04.452363 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.452105 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173ee089-1baa-4630-8a05-b994cfde37c5-istio-token\") pod \"173ee089-1baa-4630-8a05-b994cfde37c5\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " Jul 06 20:11:04.452363 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.452147 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/173ee089-1baa-4630-8a05-b994cfde37c5-istio-csr-ca-configmap\") pod \"173ee089-1baa-4630-8a05-b994cfde37c5\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " Jul 06 20:11:04.452363 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.452180 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/173ee089-1baa-4630-8a05-b994cfde37c5-local-certs\") pod \"173ee089-1baa-4630-8a05-b994cfde37c5\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " Jul 06 20:11:04.452363 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.452262 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-cacerts\") pod \"173ee089-1baa-4630-8a05-b994cfde37c5\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " Jul 06 20:11:04.452363 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.452303 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhpct\" (UniqueName: \"kubernetes.io/projected/173ee089-1baa-4630-8a05-b994cfde37c5-kube-api-access-dhpct\") pod \"173ee089-1baa-4630-8a05-b994cfde37c5\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " Jul 06 20:11:04.452363 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.452340 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-istio-csr-dns-cert\") pod \"173ee089-1baa-4630-8a05-b994cfde37c5\" (UID: \"173ee089-1baa-4630-8a05-b994cfde37c5\") " Jul 06 20:11:04.453272 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.453242 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/173ee089-1baa-4630-8a05-b994cfde37c5-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "173ee089-1baa-4630-8a05-b994cfde37c5" (UID: "173ee089-1baa-4630-8a05-b994cfde37c5"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:11:04.456588 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.456287 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "173ee089-1baa-4630-8a05-b994cfde37c5" (UID: "173ee089-1baa-4630-8a05-b994cfde37c5"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:04.456843 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.456743 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-cacerts" (OuterVolumeSpecName: "cacerts") pod "173ee089-1baa-4630-8a05-b994cfde37c5" (UID: "173ee089-1baa-4630-8a05-b994cfde37c5"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:04.457015 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.456985 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "173ee089-1baa-4630-8a05-b994cfde37c5" (UID: "173ee089-1baa-4630-8a05-b994cfde37c5"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:11:04.457390 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.457342 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/173ee089-1baa-4630-8a05-b994cfde37c5-kube-api-access-dhpct" (OuterVolumeSpecName: "kube-api-access-dhpct") pod "173ee089-1baa-4630-8a05-b994cfde37c5" (UID: "173ee089-1baa-4630-8a05-b994cfde37c5"). InnerVolumeSpecName "kube-api-access-dhpct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:11:04.458005 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.457977 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173ee089-1baa-4630-8a05-b994cfde37c5-local-certs" (OuterVolumeSpecName: "local-certs") pod "173ee089-1baa-4630-8a05-b994cfde37c5" (UID: "173ee089-1baa-4630-8a05-b994cfde37c5"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:11:04.458096 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.457979 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/173ee089-1baa-4630-8a05-b994cfde37c5-istio-token" (OuterVolumeSpecName: "istio-token") pod "173ee089-1baa-4630-8a05-b994cfde37c5" (UID: "173ee089-1baa-4630-8a05-b994cfde37c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:11:04.553617 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.553591 2583 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-cacerts\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:11:04.553617 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.553618 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhpct\" (UniqueName: \"kubernetes.io/projected/173ee089-1baa-4630-8a05-b994cfde37c5-kube-api-access-dhpct\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:11:04.553798 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.553633 2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-istio-csr-dns-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:11:04.553798 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.553642 2583 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/173ee089-1baa-4630-8a05-b994cfde37c5-istio-kubeconfig\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:11:04.553798 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.553651 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173ee089-1baa-4630-8a05-b994cfde37c5-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:11:04.553798 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.553660 2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/173ee089-1baa-4630-8a05-b994cfde37c5-istio-csr-ca-configmap\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:11:04.553798 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:04.553668 2583 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/173ee089-1baa-4630-8a05-b994cfde37c5-local-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:11:05.134177 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:05.134141 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" event={"ID":"4c49034e-79c9-407b-b068-96bbe8d61dac","Type":"ContainerStarted","Data":"2cefd9c066ddafadc66a5c40282b48dfe5c24c02b24ba6b81d7813735167847b"} Jul 06 20:11:05.134618 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:05.134210 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" Jul 06 20:11:05.135775 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:05.135746 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" event={"ID":"73340a04-d613-40d5-85ab-7c54daeaa757","Type":"ContainerStarted","Data":"afb463ca0f1bf8818ca70fa27ceb2a33e2d842f6043b88311d8474c7b53ffa4b"} Jul 06 20:11:05.135925 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:05.135802 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" Jul 06 20:11:05.137292 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:05.137272 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" event={"ID":"173ee089-1baa-4630-8a05-b994cfde37c5","Type":"ContainerDied","Data":"6a24d802ed22300b76d3bd89e89c679becb35c9acc0f054d4caa5ebc339e6b4b"} Jul 06 20:11:05.137395 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:05.137303 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb" Jul 06 20:11:05.137395 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:05.137316 2583 scope.go:117] "RemoveContainer" containerID="60b08ff115478f4eab572e9a68e6faf5cb5cb3f33dcc0a853d3321ae4e359e21" Jul 06 20:11:05.155744 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:05.155669 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" podStartSLOduration=1.8038721720000002 podStartE2EDuration="6.155655446s" totalImagesPullingTime="4.351783274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:11:05.15438929 +0000 UTC m=+318.281301393" watchObservedRunningTime="2026-07-06 20:11:05.155655446 +0000 UTC m=+318.282567579" Jul 06 20:11:05.176355 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:05.176316 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" podStartSLOduration=0.845737453 podStartE2EDuration="5.176303352s" totalImagesPullingTime="4.330565899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:11:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:11:05.174174977 +0000 UTC m=+318.301087080" watchObservedRunningTime="2026-07-06 20:11:05.176303352 +0000 UTC m=+318.303215456" Jul 06 20:11:05.197801 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:05.197779 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb"] Jul 06 20:11:05.201529 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:05.201506 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lmjnb"] Jul 06 20:11:05.484891 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:05.484851 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="173ee089-1baa-4630-8a05-b994cfde37c5" path="/var/lib/kubelet/pods/173ee089-1baa-4630-8a05-b994cfde37c5/volumes" Jul 06 20:11:06.078347 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:06.078318 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-np686" Jul 06 20:11:36.146556 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:36.146526 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" Jul 06 20:11:36.149543 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:36.149526 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" Jul 06 20:11:48.684928 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:48.684895 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-db6zp"] Jul 06 20:11:48.685289 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:48.685264 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="173ee089-1baa-4630-8a05-b994cfde37c5" containerName="discovery" Jul 06 20:11:48.685289 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:48.685279 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="173ee089-1baa-4630-8a05-b994cfde37c5" containerName="discovery" Jul 06 20:11:48.685361 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:48.685354 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="173ee089-1baa-4630-8a05-b994cfde37c5" containerName="discovery" Jul 06 20:11:48.689440 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:48.689421 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-db6zp" Jul 06 20:11:48.692359 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:48.692331 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-v6bc5\"" Jul 06 20:11:48.694088 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:48.694064 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 20:11:48.694715 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:48.694696 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 20:11:48.710410 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:48.710383 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-db6zp"] Jul 06 20:11:48.795658 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:48.795633 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j5f4\" (UniqueName: \"kubernetes.io/projected/02590153-1828-4880-83e0-59777779fc67-kube-api-access-2j5f4\") pod \"kuadrant-operator-catalog-db6zp\" (UID: \"02590153-1828-4880-83e0-59777779fc67\") " pod="kuadrant-system/kuadrant-operator-catalog-db6zp" Jul 06 20:11:48.896326 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:48.896300 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j5f4\" (UniqueName: \"kubernetes.io/projected/02590153-1828-4880-83e0-59777779fc67-kube-api-access-2j5f4\") pod \"kuadrant-operator-catalog-db6zp\" (UID: \"02590153-1828-4880-83e0-59777779fc67\") " pod="kuadrant-system/kuadrant-operator-catalog-db6zp" Jul 06 20:11:48.907895 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:48.907861 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j5f4\" (UniqueName: \"kubernetes.io/projected/02590153-1828-4880-83e0-59777779fc67-kube-api-access-2j5f4\") pod \"kuadrant-operator-catalog-db6zp\" (UID: \"02590153-1828-4880-83e0-59777779fc67\") " pod="kuadrant-system/kuadrant-operator-catalog-db6zp" Jul 06 20:11:49.000143 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:49.000116 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-db6zp"] Jul 06 20:11:49.000285 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:49.000267 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-db6zp" Jul 06 20:11:49.143311 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:49.143280 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-db6zp"] Jul 06 20:11:49.147183 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:11:49.147152 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02590153_1828_4880_83e0_59777779fc67.slice/crio-c6351955462071d8f0b24216b95d58f1fae326cb538b2b35fd5fc067906e945c WatchSource:0}: Error finding container c6351955462071d8f0b24216b95d58f1fae326cb538b2b35fd5fc067906e945c: Status 404 returned error can't find the container with id c6351955462071d8f0b24216b95d58f1fae326cb538b2b35fd5fc067906e945c Jul 06 20:11:49.219050 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:49.219023 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4khpv"] Jul 06 20:11:49.223583 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:49.223568 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4khpv" Jul 06 20:11:49.237891 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:49.237858 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4khpv"] Jul 06 20:11:49.359589 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:49.359526 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-db6zp" event={"ID":"02590153-1828-4880-83e0-59777779fc67","Type":"ContainerStarted","Data":"c6351955462071d8f0b24216b95d58f1fae326cb538b2b35fd5fc067906e945c"} Jul 06 20:11:49.400625 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:49.400601 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgch5\" (UniqueName: \"kubernetes.io/projected/d4d809a6-2593-4470-b8a9-49c663b07b0e-kube-api-access-rgch5\") pod \"kuadrant-operator-catalog-4khpv\" (UID: \"d4d809a6-2593-4470-b8a9-49c663b07b0e\") " pod="kuadrant-system/kuadrant-operator-catalog-4khpv" Jul 06 20:11:49.501205 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:49.501180 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgch5\" (UniqueName: \"kubernetes.io/projected/d4d809a6-2593-4470-b8a9-49c663b07b0e-kube-api-access-rgch5\") pod \"kuadrant-operator-catalog-4khpv\" (UID: \"d4d809a6-2593-4470-b8a9-49c663b07b0e\") " pod="kuadrant-system/kuadrant-operator-catalog-4khpv" Jul 06 20:11:49.513250 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:49.513225 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgch5\" (UniqueName: \"kubernetes.io/projected/d4d809a6-2593-4470-b8a9-49c663b07b0e-kube-api-access-rgch5\") pod \"kuadrant-operator-catalog-4khpv\" (UID: \"d4d809a6-2593-4470-b8a9-49c663b07b0e\") " pod="kuadrant-system/kuadrant-operator-catalog-4khpv" Jul 06 20:11:49.532841 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:49.532817 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4khpv" Jul 06 20:11:49.671776 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:49.671749 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4khpv"] Jul 06 20:11:49.673650 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:11:49.673619 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d809a6_2593_4470_b8a9_49c663b07b0e.slice/crio-ce836ba11631890d9eecb6bd51c55c77c3adcf928bfed71e0c0e582a487975a4 WatchSource:0}: Error finding container ce836ba11631890d9eecb6bd51c55c77c3adcf928bfed71e0c0e582a487975a4: Status 404 returned error can't find the container with id ce836ba11631890d9eecb6bd51c55c77c3adcf928bfed71e0c0e582a487975a4 Jul 06 20:11:50.367724 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:50.367691 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4khpv" event={"ID":"d4d809a6-2593-4470-b8a9-49c663b07b0e","Type":"ContainerStarted","Data":"ce836ba11631890d9eecb6bd51c55c77c3adcf928bfed71e0c0e582a487975a4"} Jul 06 20:11:52.379674 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:52.379636 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4khpv" event={"ID":"d4d809a6-2593-4470-b8a9-49c663b07b0e","Type":"ContainerStarted","Data":"c1dd3bd98304268a397516e4395a0e4dd6037ffe353688ed4bb77e1be7c734cd"} Jul 06 20:11:52.381193 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:52.381162 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-db6zp" event={"ID":"02590153-1828-4880-83e0-59777779fc67","Type":"ContainerStarted","Data":"ef5fe25b21232e4f412568cc2ffb4a9ce8836dd2c630a2870d4345747f67065e"} Jul 06 20:11:52.381332 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:52.381190 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-db6zp" podUID="02590153-1828-4880-83e0-59777779fc67" containerName="registry-server" containerID="cri-o://ef5fe25b21232e4f412568cc2ffb4a9ce8836dd2c630a2870d4345747f67065e" gracePeriod=2 Jul 06 20:11:52.401449 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:52.401413 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4khpv" podStartSLOduration=1.647079015 podStartE2EDuration="3.401399996s" totalImagesPullingTime="1.754320981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:11:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:11:52.399896023 +0000 UTC m=+365.526808125" watchObservedRunningTime="2026-07-06 20:11:52.401399996 +0000 UTC m=+365.528312110" Jul 06 20:11:52.418584 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:52.418537 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-db6zp" podStartSLOduration=2.142473534 podStartE2EDuration="4.418524563s" totalImagesPullingTime="2.276051029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:11:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:11:52.417462786 +0000 UTC m=+365.544374888" watchObservedRunningTime="2026-07-06 20:11:52.418524563 +0000 UTC m=+365.545436667" Jul 06 20:11:52.627060 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:52.627038 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-db6zp" Jul 06 20:11:52.727699 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:52.727676 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2j5f4\" (UniqueName: \"kubernetes.io/projected/02590153-1828-4880-83e0-59777779fc67-kube-api-access-2j5f4\") pod \"02590153-1828-4880-83e0-59777779fc67\" (UID: \"02590153-1828-4880-83e0-59777779fc67\") " Jul 06 20:11:52.729762 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:52.729733 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02590153-1828-4880-83e0-59777779fc67-kube-api-access-2j5f4" (OuterVolumeSpecName: "kube-api-access-2j5f4") pod "02590153-1828-4880-83e0-59777779fc67" (UID: "02590153-1828-4880-83e0-59777779fc67"). InnerVolumeSpecName "kube-api-access-2j5f4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:11:52.828500 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:52.828476 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2j5f4\" (UniqueName: \"kubernetes.io/projected/02590153-1828-4880-83e0-59777779fc67-kube-api-access-2j5f4\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:11:53.387537 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:53.387498 2583 generic.go:356] "Generic (PLEG): container finished" podID="02590153-1828-4880-83e0-59777779fc67" containerID="ef5fe25b21232e4f412568cc2ffb4a9ce8836dd2c630a2870d4345747f67065e" exitCode=0 Jul 06 20:11:53.387992 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:53.387547 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-db6zp" Jul 06 20:11:53.387992 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:53.387587 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-db6zp" event={"ID":"02590153-1828-4880-83e0-59777779fc67","Type":"ContainerDied","Data":"ef5fe25b21232e4f412568cc2ffb4a9ce8836dd2c630a2870d4345747f67065e"} Jul 06 20:11:53.387992 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:53.387646 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-db6zp" event={"ID":"02590153-1828-4880-83e0-59777779fc67","Type":"ContainerDied","Data":"c6351955462071d8f0b24216b95d58f1fae326cb538b2b35fd5fc067906e945c"} Jul 06 20:11:53.387992 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:53.387678 2583 scope.go:117] "RemoveContainer" containerID="ef5fe25b21232e4f412568cc2ffb4a9ce8836dd2c630a2870d4345747f67065e" Jul 06 20:11:53.398100 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:53.398079 2583 scope.go:117] "RemoveContainer" containerID="ef5fe25b21232e4f412568cc2ffb4a9ce8836dd2c630a2870d4345747f67065e" Jul 06 20:11:53.398349 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:11:53.398331 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef5fe25b21232e4f412568cc2ffb4a9ce8836dd2c630a2870d4345747f67065e\": container with ID starting with ef5fe25b21232e4f412568cc2ffb4a9ce8836dd2c630a2870d4345747f67065e not found: ID does not exist" containerID="ef5fe25b21232e4f412568cc2ffb4a9ce8836dd2c630a2870d4345747f67065e" Jul 06 20:11:53.398412 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:53.398356 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef5fe25b21232e4f412568cc2ffb4a9ce8836dd2c630a2870d4345747f67065e"} err="failed to get container status \"ef5fe25b21232e4f412568cc2ffb4a9ce8836dd2c630a2870d4345747f67065e\": rpc error: code = NotFound desc = could not find container \"ef5fe25b21232e4f412568cc2ffb4a9ce8836dd2c630a2870d4345747f67065e\": container with ID starting with ef5fe25b21232e4f412568cc2ffb4a9ce8836dd2c630a2870d4345747f67065e not found: ID does not exist" Jul 06 20:11:53.414259 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:53.414234 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-db6zp"] Jul 06 20:11:53.419033 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:53.419011 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-db6zp"] Jul 06 20:11:53.484333 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:53.484311 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02590153-1828-4880-83e0-59777779fc67" path="/var/lib/kubelet/pods/02590153-1828-4880-83e0-59777779fc67/volumes" Jul 06 20:11:59.533797 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:59.533769 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-4khpv" Jul 06 20:11:59.534306 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:59.533850 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-4khpv" Jul 06 20:11:59.556282 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:11:59.556258 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-4khpv" Jul 06 20:12:00.446416 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:00.446383 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-4khpv" Jul 06 20:12:04.259183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.259155 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6"] Jul 06 20:12:04.259574 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.259522 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02590153-1828-4880-83e0-59777779fc67" containerName="registry-server" Jul 06 20:12:04.259574 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.259536 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="02590153-1828-4880-83e0-59777779fc67" containerName="registry-server" Jul 06 20:12:04.259670 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.259603 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="02590153-1828-4880-83e0-59777779fc67" containerName="registry-server" Jul 06 20:12:04.264438 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.264417 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" Jul 06 20:12:04.267224 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.267205 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fgg7s\"" Jul 06 20:12:04.274096 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.274071 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6"] Jul 06 20:12:04.304117 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.304081 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbmsg\" (UniqueName: \"kubernetes.io/projected/6603e100-ee5e-4c3d-9063-0b638e10b392-kube-api-access-cbmsg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6\" (UID: \"6603e100-ee5e-4c3d-9063-0b638e10b392\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" Jul 06 20:12:04.304223 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.304154 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6603e100-ee5e-4c3d-9063-0b638e10b392-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6\" (UID: \"6603e100-ee5e-4c3d-9063-0b638e10b392\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" Jul 06 20:12:04.304281 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.304232 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6603e100-ee5e-4c3d-9063-0b638e10b392-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6\" (UID: \"6603e100-ee5e-4c3d-9063-0b638e10b392\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" Jul 06 20:12:04.405144 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.405118 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbmsg\" (UniqueName: \"kubernetes.io/projected/6603e100-ee5e-4c3d-9063-0b638e10b392-kube-api-access-cbmsg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6\" (UID: \"6603e100-ee5e-4c3d-9063-0b638e10b392\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" Jul 06 20:12:04.405256 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.405149 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6603e100-ee5e-4c3d-9063-0b638e10b392-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6\" (UID: \"6603e100-ee5e-4c3d-9063-0b638e10b392\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" Jul 06 20:12:04.405256 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.405189 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6603e100-ee5e-4c3d-9063-0b638e10b392-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6\" (UID: \"6603e100-ee5e-4c3d-9063-0b638e10b392\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" Jul 06 20:12:04.405501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.405482 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6603e100-ee5e-4c3d-9063-0b638e10b392-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6\" (UID: \"6603e100-ee5e-4c3d-9063-0b638e10b392\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" Jul 06 20:12:04.405563 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.405500 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6603e100-ee5e-4c3d-9063-0b638e10b392-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6\" (UID: \"6603e100-ee5e-4c3d-9063-0b638e10b392\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" Jul 06 20:12:04.421095 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.421070 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbmsg\" (UniqueName: \"kubernetes.io/projected/6603e100-ee5e-4c3d-9063-0b638e10b392-kube-api-access-cbmsg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6\" (UID: \"6603e100-ee5e-4c3d-9063-0b638e10b392\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" Jul 06 20:12:04.573786 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.573736 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" Jul 06 20:12:04.709272 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:04.709243 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6"] Jul 06 20:12:04.710195 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:12:04.710170 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6603e100_ee5e_4c3d_9063_0b638e10b392.slice/crio-b52851c03a9561e402519333cfe1b07dd5ed63112b1c2cd99c6d937404a41013 WatchSource:0}: Error finding container b52851c03a9561e402519333cfe1b07dd5ed63112b1c2cd99c6d937404a41013: Status 404 returned error can't find the container with id b52851c03a9561e402519333cfe1b07dd5ed63112b1c2cd99c6d937404a41013 Jul 06 20:12:05.067985 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.067952 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5"] Jul 06 20:12:05.071663 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.071643 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" Jul 06 20:12:05.081986 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.081959 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5"] Jul 06 20:12:05.111681 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.111655 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5\" (UID: \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" Jul 06 20:12:05.111790 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.111694 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5\" (UID: \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" Jul 06 20:12:05.111841 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.111819 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8n7f\" (UniqueName: \"kubernetes.io/projected/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-kube-api-access-x8n7f\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5\" (UID: \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" Jul 06 20:12:05.212647 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.212607 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8n7f\" (UniqueName: \"kubernetes.io/projected/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-kube-api-access-x8n7f\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5\" (UID: \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" Jul 06 20:12:05.212769 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.212708 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5\" (UID: \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" Jul 06 20:12:05.212769 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.212743 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5\" (UID: \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" Jul 06 20:12:05.213093 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.213074 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5\" (UID: \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" Jul 06 20:12:05.213168 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.213112 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5\" (UID: \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" Jul 06 20:12:05.227818 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.227794 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8n7f\" (UniqueName: \"kubernetes.io/projected/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-kube-api-access-x8n7f\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5\" (UID: \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" Jul 06 20:12:05.381242 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.381187 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" Jul 06 20:12:05.451158 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.451129 2583 generic.go:356] "Generic (PLEG): container finished" podID="6603e100-ee5e-4c3d-9063-0b638e10b392" containerID="e01fdbec548b5f601457ce1635a2b63e01f2549b4e9e09464c8acc7533f1705a" exitCode=0 Jul 06 20:12:05.451284 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.451171 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" event={"ID":"6603e100-ee5e-4c3d-9063-0b638e10b392","Type":"ContainerDied","Data":"e01fdbec548b5f601457ce1635a2b63e01f2549b4e9e09464c8acc7533f1705a"} Jul 06 20:12:05.451284 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.451237 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" event={"ID":"6603e100-ee5e-4c3d-9063-0b638e10b392","Type":"ContainerStarted","Data":"b52851c03a9561e402519333cfe1b07dd5ed63112b1c2cd99c6d937404a41013"} Jul 06 20:12:05.532391 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.532365 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5"] Jul 06 20:12:05.533345 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:12:05.533323 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod602cda8d_f8c4_4a6a_9a04_5759fc1af23c.slice/crio-2902f838ed32d43545da09840579ef4c51d8b28d475806706f8027ef1a0524fc WatchSource:0}: Error finding container 2902f838ed32d43545da09840579ef4c51d8b28d475806706f8027ef1a0524fc: Status 404 returned error can't find the container with id 2902f838ed32d43545da09840579ef4c51d8b28d475806706f8027ef1a0524fc Jul 06 20:12:05.676210 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.676178 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd"] Jul 06 20:12:05.679677 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.679660 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" Jul 06 20:12:05.690458 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.690432 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd"] Jul 06 20:12:05.716434 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.716409 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vc8b\" (UniqueName: \"kubernetes.io/projected/5884ee3f-4633-4425-a422-abcb819e01b5-kube-api-access-7vc8b\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd\" (UID: \"5884ee3f-4633-4425-a422-abcb819e01b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" Jul 06 20:12:05.716532 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.716479 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5884ee3f-4633-4425-a422-abcb819e01b5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd\" (UID: \"5884ee3f-4633-4425-a422-abcb819e01b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" Jul 06 20:12:05.716571 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.716543 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5884ee3f-4633-4425-a422-abcb819e01b5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd\" (UID: \"5884ee3f-4633-4425-a422-abcb819e01b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" Jul 06 20:12:05.818005 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.817971 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5884ee3f-4633-4425-a422-abcb819e01b5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd\" (UID: \"5884ee3f-4633-4425-a422-abcb819e01b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" Jul 06 20:12:05.818157 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.818026 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5884ee3f-4633-4425-a422-abcb819e01b5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd\" (UID: \"5884ee3f-4633-4425-a422-abcb819e01b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" Jul 06 20:12:05.818157 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.818068 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vc8b\" (UniqueName: \"kubernetes.io/projected/5884ee3f-4633-4425-a422-abcb819e01b5-kube-api-access-7vc8b\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd\" (UID: \"5884ee3f-4633-4425-a422-abcb819e01b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" Jul 06 20:12:05.818389 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.818367 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5884ee3f-4633-4425-a422-abcb819e01b5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd\" (UID: \"5884ee3f-4633-4425-a422-abcb819e01b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" Jul 06 20:12:05.818389 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.818379 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5884ee3f-4633-4425-a422-abcb819e01b5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd\" (UID: \"5884ee3f-4633-4425-a422-abcb819e01b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" Jul 06 20:12:05.830034 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.830014 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vc8b\" (UniqueName: \"kubernetes.io/projected/5884ee3f-4633-4425-a422-abcb819e01b5-kube-api-access-7vc8b\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd\" (UID: \"5884ee3f-4633-4425-a422-abcb819e01b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" Jul 06 20:12:05.996907 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:05.996865 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" Jul 06 20:12:06.129919 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.129894 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd"] Jul 06 20:12:06.131631 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:12:06.131604 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5884ee3f_4633_4425_a422_abcb819e01b5.slice/crio-daaaf3aabc5ee0a1edd6cd04f6ee6bef03e7f6f64cbbbeb040092f2c712a2ab7 WatchSource:0}: Error finding container daaaf3aabc5ee0a1edd6cd04f6ee6bef03e7f6f64cbbbeb040092f2c712a2ab7: Status 404 returned error can't find the container with id daaaf3aabc5ee0a1edd6cd04f6ee6bef03e7f6f64cbbbeb040092f2c712a2ab7 Jul 06 20:12:06.265074 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.265049 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96"] Jul 06 20:12:06.269738 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.269721 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" Jul 06 20:12:06.279571 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.279546 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96"] Jul 06 20:12:06.323045 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.323019 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6485c000-68a5-4ee0-a899-9925bee9b133-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96\" (UID: \"6485c000-68a5-4ee0-a899-9925bee9b133\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" Jul 06 20:12:06.323140 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.323076 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6485c000-68a5-4ee0-a899-9925bee9b133-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96\" (UID: \"6485c000-68a5-4ee0-a899-9925bee9b133\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" Jul 06 20:12:06.323195 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.323183 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cqjh\" (UniqueName: \"kubernetes.io/projected/6485c000-68a5-4ee0-a899-9925bee9b133-kube-api-access-6cqjh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96\" (UID: \"6485c000-68a5-4ee0-a899-9925bee9b133\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" Jul 06 20:12:06.424079 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.424051 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cqjh\" (UniqueName: \"kubernetes.io/projected/6485c000-68a5-4ee0-a899-9925bee9b133-kube-api-access-6cqjh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96\" (UID: \"6485c000-68a5-4ee0-a899-9925bee9b133\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" Jul 06 20:12:06.424365 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.424084 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6485c000-68a5-4ee0-a899-9925bee9b133-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96\" (UID: \"6485c000-68a5-4ee0-a899-9925bee9b133\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" Jul 06 20:12:06.424365 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.424107 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6485c000-68a5-4ee0-a899-9925bee9b133-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96\" (UID: \"6485c000-68a5-4ee0-a899-9925bee9b133\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" Jul 06 20:12:06.424454 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.424436 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6485c000-68a5-4ee0-a899-9925bee9b133-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96\" (UID: \"6485c000-68a5-4ee0-a899-9925bee9b133\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" Jul 06 20:12:06.424490 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.424450 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6485c000-68a5-4ee0-a899-9925bee9b133-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96\" (UID: \"6485c000-68a5-4ee0-a899-9925bee9b133\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" Jul 06 20:12:06.435101 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.435080 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cqjh\" (UniqueName: \"kubernetes.io/projected/6485c000-68a5-4ee0-a899-9925bee9b133-kube-api-access-6cqjh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96\" (UID: \"6485c000-68a5-4ee0-a899-9925bee9b133\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" Jul 06 20:12:06.457611 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.457589 2583 generic.go:356] "Generic (PLEG): container finished" podID="6603e100-ee5e-4c3d-9063-0b638e10b392" containerID="5f4f5971429f67c05c9393c60a3bcaeee5b3fc22068129297f98c3c29074a662" exitCode=0 Jul 06 20:12:06.457713 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.457674 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" event={"ID":"6603e100-ee5e-4c3d-9063-0b638e10b392","Type":"ContainerDied","Data":"5f4f5971429f67c05c9393c60a3bcaeee5b3fc22068129297f98c3c29074a662"} Jul 06 20:12:06.459478 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.459458 2583 generic.go:356] "Generic (PLEG): container finished" podID="5884ee3f-4633-4425-a422-abcb819e01b5" containerID="977d6b48b87543ab8d6488b4f01d1edd096967527784942f9d467e594a72536b" exitCode=0 Jul 06 20:12:06.459564 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.459488 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" event={"ID":"5884ee3f-4633-4425-a422-abcb819e01b5","Type":"ContainerDied","Data":"977d6b48b87543ab8d6488b4f01d1edd096967527784942f9d467e594a72536b"} Jul 06 20:12:06.459564 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.459523 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" event={"ID":"5884ee3f-4633-4425-a422-abcb819e01b5","Type":"ContainerStarted","Data":"daaaf3aabc5ee0a1edd6cd04f6ee6bef03e7f6f64cbbbeb040092f2c712a2ab7"} Jul 06 20:12:06.461278 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.461256 2583 generic.go:356] "Generic (PLEG): container finished" podID="602cda8d-f8c4-4a6a-9a04-5759fc1af23c" containerID="f2ad1bc9e3abe5c9db6f7c441dde6381d8bee1b25d308ac5e2fcca714e9a2ff9" exitCode=0 Jul 06 20:12:06.461371 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.461282 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" event={"ID":"602cda8d-f8c4-4a6a-9a04-5759fc1af23c","Type":"ContainerDied","Data":"f2ad1bc9e3abe5c9db6f7c441dde6381d8bee1b25d308ac5e2fcca714e9a2ff9"} Jul 06 20:12:06.461371 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.461337 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" event={"ID":"602cda8d-f8c4-4a6a-9a04-5759fc1af23c","Type":"ContainerStarted","Data":"2902f838ed32d43545da09840579ef4c51d8b28d475806706f8027ef1a0524fc"} Jul 06 20:12:06.655263 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.655241 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" Jul 06 20:12:06.790237 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:06.790211 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96"] Jul 06 20:12:06.792651 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:12:06.792623 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6485c000_68a5_4ee0_a899_9925bee9b133.slice/crio-561d8d6ca52172995a6db938e53303068b87998c3bb24ca3825134c2b2315700 WatchSource:0}: Error finding container 561d8d6ca52172995a6db938e53303068b87998c3bb24ca3825134c2b2315700: Status 404 returned error can't find the container with id 561d8d6ca52172995a6db938e53303068b87998c3bb24ca3825134c2b2315700 Jul 06 20:12:07.468605 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:07.468577 2583 generic.go:356] "Generic (PLEG): container finished" podID="6603e100-ee5e-4c3d-9063-0b638e10b392" containerID="5e29faba3285300664f627314f79ab4b550aa66997e83d2e5a8ee61cab562b73" exitCode=0 Jul 06 20:12:07.468958 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:07.468644 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" event={"ID":"6603e100-ee5e-4c3d-9063-0b638e10b392","Type":"ContainerDied","Data":"5e29faba3285300664f627314f79ab4b550aa66997e83d2e5a8ee61cab562b73"} Jul 06 20:12:07.470639 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:07.470614 2583 generic.go:356] "Generic (PLEG): container finished" podID="5884ee3f-4633-4425-a422-abcb819e01b5" containerID="83e52c40da2748c82036df4d7c585709a57aa8043086fda1e3e1b855ae3bf11f" exitCode=0 Jul 06 20:12:07.470736 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:07.470694 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" event={"ID":"5884ee3f-4633-4425-a422-abcb819e01b5","Type":"ContainerDied","Data":"83e52c40da2748c82036df4d7c585709a57aa8043086fda1e3e1b855ae3bf11f"} Jul 06 20:12:07.472246 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:07.472225 2583 generic.go:356] "Generic (PLEG): container finished" podID="6485c000-68a5-4ee0-a899-9925bee9b133" containerID="bb7ed553a099807ce3ccbbd8c9f8426879ecf0ff0e479e5f76b6b5e3b399a0c0" exitCode=0 Jul 06 20:12:07.472322 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:07.472302 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" event={"ID":"6485c000-68a5-4ee0-a899-9925bee9b133","Type":"ContainerDied","Data":"bb7ed553a099807ce3ccbbd8c9f8426879ecf0ff0e479e5f76b6b5e3b399a0c0"} Jul 06 20:12:07.472386 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:07.472354 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" event={"ID":"6485c000-68a5-4ee0-a899-9925bee9b133","Type":"ContainerStarted","Data":"561d8d6ca52172995a6db938e53303068b87998c3bb24ca3825134c2b2315700"} Jul 06 20:12:07.474464 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:07.474443 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" event={"ID":"602cda8d-f8c4-4a6a-9a04-5759fc1af23c","Type":"ContainerStarted","Data":"c30ca8b2966b0a5f223ed44d196dd5adb954d76e7bff9433ee0b237dc7e5ac2d"} Jul 06 20:12:08.481532 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.481508 2583 generic.go:356] "Generic (PLEG): container finished" podID="5884ee3f-4633-4425-a422-abcb819e01b5" containerID="dee5a6a2ee80d48760b980835597e658c36a13e8811fd68faf2243b57006d424" exitCode=0 Jul 06 20:12:08.481890 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.481575 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" event={"ID":"5884ee3f-4633-4425-a422-abcb819e01b5","Type":"ContainerDied","Data":"dee5a6a2ee80d48760b980835597e658c36a13e8811fd68faf2243b57006d424"} Jul 06 20:12:08.483462 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.483441 2583 generic.go:356] "Generic (PLEG): container finished" podID="6485c000-68a5-4ee0-a899-9925bee9b133" containerID="6e87432ff669fdef87c9903b7041b3d383b4c1209cfc9fc72d505441356c83ca" exitCode=0 Jul 06 20:12:08.483556 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.483523 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" event={"ID":"6485c000-68a5-4ee0-a899-9925bee9b133","Type":"ContainerDied","Data":"6e87432ff669fdef87c9903b7041b3d383b4c1209cfc9fc72d505441356c83ca"} Jul 06 20:12:08.485430 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.485406 2583 generic.go:356] "Generic (PLEG): container finished" podID="602cda8d-f8c4-4a6a-9a04-5759fc1af23c" containerID="c30ca8b2966b0a5f223ed44d196dd5adb954d76e7bff9433ee0b237dc7e5ac2d" exitCode=0 Jul 06 20:12:08.485528 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.485486 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" event={"ID":"602cda8d-f8c4-4a6a-9a04-5759fc1af23c","Type":"ContainerDied","Data":"c30ca8b2966b0a5f223ed44d196dd5adb954d76e7bff9433ee0b237dc7e5ac2d"} Jul 06 20:12:08.631244 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.631207 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" Jul 06 20:12:08.643216 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.643189 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6603e100-ee5e-4c3d-9063-0b638e10b392-bundle\") pod \"6603e100-ee5e-4c3d-9063-0b638e10b392\" (UID: \"6603e100-ee5e-4c3d-9063-0b638e10b392\") " Jul 06 20:12:08.643358 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.643259 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbmsg\" (UniqueName: \"kubernetes.io/projected/6603e100-ee5e-4c3d-9063-0b638e10b392-kube-api-access-cbmsg\") pod \"6603e100-ee5e-4c3d-9063-0b638e10b392\" (UID: \"6603e100-ee5e-4c3d-9063-0b638e10b392\") " Jul 06 20:12:08.643358 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.643309 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6603e100-ee5e-4c3d-9063-0b638e10b392-util\") pod \"6603e100-ee5e-4c3d-9063-0b638e10b392\" (UID: \"6603e100-ee5e-4c3d-9063-0b638e10b392\") " Jul 06 20:12:08.644326 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.644298 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6603e100-ee5e-4c3d-9063-0b638e10b392-bundle" (OuterVolumeSpecName: "bundle") pod "6603e100-ee5e-4c3d-9063-0b638e10b392" (UID: "6603e100-ee5e-4c3d-9063-0b638e10b392"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:12:08.645564 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.645544 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6603e100-ee5e-4c3d-9063-0b638e10b392-kube-api-access-cbmsg" (OuterVolumeSpecName: "kube-api-access-cbmsg") pod "6603e100-ee5e-4c3d-9063-0b638e10b392" (UID: "6603e100-ee5e-4c3d-9063-0b638e10b392"). InnerVolumeSpecName "kube-api-access-cbmsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:12:08.649894 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.649664 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6603e100-ee5e-4c3d-9063-0b638e10b392-util" (OuterVolumeSpecName: "util") pod "6603e100-ee5e-4c3d-9063-0b638e10b392" (UID: "6603e100-ee5e-4c3d-9063-0b638e10b392"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:12:08.744580 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.744517 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6603e100-ee5e-4c3d-9063-0b638e10b392-bundle\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:08.744580 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.744544 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbmsg\" (UniqueName: \"kubernetes.io/projected/6603e100-ee5e-4c3d-9063-0b638e10b392-kube-api-access-cbmsg\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:08.744580 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:08.744557 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6603e100-ee5e-4c3d-9063-0b638e10b392-util\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:09.499030 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.498998 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" event={"ID":"6603e100-ee5e-4c3d-9063-0b638e10b392","Type":"ContainerDied","Data":"b52851c03a9561e402519333cfe1b07dd5ed63112b1c2cd99c6d937404a41013"} Jul 06 20:12:09.499384 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.499070 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b52851c03a9561e402519333cfe1b07dd5ed63112b1c2cd99c6d937404a41013" Jul 06 20:12:09.499384 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.499014 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6" Jul 06 20:12:09.501466 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.501444 2583 generic.go:356] "Generic (PLEG): container finished" podID="6485c000-68a5-4ee0-a899-9925bee9b133" containerID="af1568cf9f9436d617ee10bf246607de245ac8275b69cbb29e692040c72b6f14" exitCode=0 Jul 06 20:12:09.501594 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.501532 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" event={"ID":"6485c000-68a5-4ee0-a899-9925bee9b133","Type":"ContainerDied","Data":"af1568cf9f9436d617ee10bf246607de245ac8275b69cbb29e692040c72b6f14"} Jul 06 20:12:09.503633 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.503613 2583 generic.go:356] "Generic (PLEG): container finished" podID="602cda8d-f8c4-4a6a-9a04-5759fc1af23c" containerID="7eb8e46b1caa9f1f65652fc0a8140187277bc291977d7fa5ae7e69f57961cea1" exitCode=0 Jul 06 20:12:09.503744 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.503700 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" event={"ID":"602cda8d-f8c4-4a6a-9a04-5759fc1af23c","Type":"ContainerDied","Data":"7eb8e46b1caa9f1f65652fc0a8140187277bc291977d7fa5ae7e69f57961cea1"} Jul 06 20:12:09.633442 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.633419 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" Jul 06 20:12:09.651238 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.651215 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5884ee3f-4633-4425-a422-abcb819e01b5-bundle\") pod \"5884ee3f-4633-4425-a422-abcb819e01b5\" (UID: \"5884ee3f-4633-4425-a422-abcb819e01b5\") " Jul 06 20:12:09.651359 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.651249 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5884ee3f-4633-4425-a422-abcb819e01b5-util\") pod \"5884ee3f-4633-4425-a422-abcb819e01b5\" (UID: \"5884ee3f-4633-4425-a422-abcb819e01b5\") " Jul 06 20:12:09.651359 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.651298 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vc8b\" (UniqueName: \"kubernetes.io/projected/5884ee3f-4633-4425-a422-abcb819e01b5-kube-api-access-7vc8b\") pod \"5884ee3f-4633-4425-a422-abcb819e01b5\" (UID: \"5884ee3f-4633-4425-a422-abcb819e01b5\") " Jul 06 20:12:09.651686 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.651663 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5884ee3f-4633-4425-a422-abcb819e01b5-bundle" (OuterVolumeSpecName: "bundle") pod "5884ee3f-4633-4425-a422-abcb819e01b5" (UID: "5884ee3f-4633-4425-a422-abcb819e01b5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:12:09.653755 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.653718 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5884ee3f-4633-4425-a422-abcb819e01b5-kube-api-access-7vc8b" (OuterVolumeSpecName: "kube-api-access-7vc8b") pod "5884ee3f-4633-4425-a422-abcb819e01b5" (UID: "5884ee3f-4633-4425-a422-abcb819e01b5"). InnerVolumeSpecName "kube-api-access-7vc8b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:12:09.657912 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.657857 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5884ee3f-4633-4425-a422-abcb819e01b5-util" (OuterVolumeSpecName: "util") pod "5884ee3f-4633-4425-a422-abcb819e01b5" (UID: "5884ee3f-4633-4425-a422-abcb819e01b5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:12:09.752264 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.752193 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5884ee3f-4633-4425-a422-abcb819e01b5-bundle\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:09.752264 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.752217 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5884ee3f-4633-4425-a422-abcb819e01b5-util\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:09.752264 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:09.752226 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vc8b\" (UniqueName: \"kubernetes.io/projected/5884ee3f-4633-4425-a422-abcb819e01b5-kube-api-access-7vc8b\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:10.511452 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.511420 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" Jul 06 20:12:10.511452 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.511438 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd" event={"ID":"5884ee3f-4633-4425-a422-abcb819e01b5","Type":"ContainerDied","Data":"daaaf3aabc5ee0a1edd6cd04f6ee6bef03e7f6f64cbbbeb040092f2c712a2ab7"} Jul 06 20:12:10.511869 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.511480 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="daaaf3aabc5ee0a1edd6cd04f6ee6bef03e7f6f64cbbbeb040092f2c712a2ab7" Jul 06 20:12:10.650792 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.650769 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" Jul 06 20:12:10.680791 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.680770 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" Jul 06 20:12:10.761822 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.761757 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8n7f\" (UniqueName: \"kubernetes.io/projected/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-kube-api-access-x8n7f\") pod \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\" (UID: \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\") " Jul 06 20:12:10.761822 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.761799 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6485c000-68a5-4ee0-a899-9925bee9b133-bundle\") pod \"6485c000-68a5-4ee0-a899-9925bee9b133\" (UID: \"6485c000-68a5-4ee0-a899-9925bee9b133\") " Jul 06 20:12:10.762129 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.761829 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-bundle\") pod \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\" (UID: \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\") " Jul 06 20:12:10.762129 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.761866 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cqjh\" (UniqueName: \"kubernetes.io/projected/6485c000-68a5-4ee0-a899-9925bee9b133-kube-api-access-6cqjh\") pod \"6485c000-68a5-4ee0-a899-9925bee9b133\" (UID: \"6485c000-68a5-4ee0-a899-9925bee9b133\") " Jul 06 20:12:10.762129 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.761900 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-util\") pod \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\" (UID: \"602cda8d-f8c4-4a6a-9a04-5759fc1af23c\") " Jul 06 20:12:10.762129 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.761922 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6485c000-68a5-4ee0-a899-9925bee9b133-util\") pod \"6485c000-68a5-4ee0-a899-9925bee9b133\" (UID: \"6485c000-68a5-4ee0-a899-9925bee9b133\") " Jul 06 20:12:10.762642 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.762471 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-bundle" (OuterVolumeSpecName: "bundle") pod "602cda8d-f8c4-4a6a-9a04-5759fc1af23c" (UID: "602cda8d-f8c4-4a6a-9a04-5759fc1af23c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:12:10.762740 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.762684 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6485c000-68a5-4ee0-a899-9925bee9b133-bundle" (OuterVolumeSpecName: "bundle") pod "6485c000-68a5-4ee0-a899-9925bee9b133" (UID: "6485c000-68a5-4ee0-a899-9925bee9b133"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:12:10.764087 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.764062 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-kube-api-access-x8n7f" (OuterVolumeSpecName: "kube-api-access-x8n7f") pod "602cda8d-f8c4-4a6a-9a04-5759fc1af23c" (UID: "602cda8d-f8c4-4a6a-9a04-5759fc1af23c"). InnerVolumeSpecName "kube-api-access-x8n7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:12:10.764170 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.764118 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6485c000-68a5-4ee0-a899-9925bee9b133-kube-api-access-6cqjh" (OuterVolumeSpecName: "kube-api-access-6cqjh") pod "6485c000-68a5-4ee0-a899-9925bee9b133" (UID: "6485c000-68a5-4ee0-a899-9925bee9b133"). InnerVolumeSpecName "kube-api-access-6cqjh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:12:10.767950 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.767922 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6485c000-68a5-4ee0-a899-9925bee9b133-util" (OuterVolumeSpecName: "util") pod "6485c000-68a5-4ee0-a899-9925bee9b133" (UID: "6485c000-68a5-4ee0-a899-9925bee9b133"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:12:10.767950 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.767929 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-util" (OuterVolumeSpecName: "util") pod "602cda8d-f8c4-4a6a-9a04-5759fc1af23c" (UID: "602cda8d-f8c4-4a6a-9a04-5759fc1af23c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:12:10.863095 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.863058 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cqjh\" (UniqueName: \"kubernetes.io/projected/6485c000-68a5-4ee0-a899-9925bee9b133-kube-api-access-6cqjh\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:10.863095 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.863094 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-util\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:10.863095 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.863105 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6485c000-68a5-4ee0-a899-9925bee9b133-util\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:10.863258 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.863113 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8n7f\" (UniqueName: \"kubernetes.io/projected/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-kube-api-access-x8n7f\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:10.863258 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.863122 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6485c000-68a5-4ee0-a899-9925bee9b133-bundle\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:10.863258 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:10.863130 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/602cda8d-f8c4-4a6a-9a04-5759fc1af23c-bundle\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:11.518831 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:11.518808 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" Jul 06 20:12:11.518831 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:11.518818 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96" event={"ID":"6485c000-68a5-4ee0-a899-9925bee9b133","Type":"ContainerDied","Data":"561d8d6ca52172995a6db938e53303068b87998c3bb24ca3825134c2b2315700"} Jul 06 20:12:11.519267 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:11.518854 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="561d8d6ca52172995a6db938e53303068b87998c3bb24ca3825134c2b2315700" Jul 06 20:12:11.521029 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:11.521004 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" event={"ID":"602cda8d-f8c4-4a6a-9a04-5759fc1af23c","Type":"ContainerDied","Data":"2902f838ed32d43545da09840579ef4c51d8b28d475806706f8027ef1a0524fc"} Jul 06 20:12:11.521131 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:11.521043 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2902f838ed32d43545da09840579ef4c51d8b28d475806706f8027ef1a0524fc" Jul 06 20:12:11.521131 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:11.521055 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5" Jul 06 20:12:20.813708 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.813673 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch"] Jul 06 20:12:20.814091 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814024 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6603e100-ee5e-4c3d-9063-0b638e10b392" containerName="util" Jul 06 20:12:20.814091 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814035 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6603e100-ee5e-4c3d-9063-0b638e10b392" containerName="util" Jul 06 20:12:20.814091 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814046 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="602cda8d-f8c4-4a6a-9a04-5759fc1af23c" containerName="util" Jul 06 20:12:20.814091 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814051 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="602cda8d-f8c4-4a6a-9a04-5759fc1af23c" containerName="util" Jul 06 20:12:20.814091 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814058 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="602cda8d-f8c4-4a6a-9a04-5759fc1af23c" containerName="pull" Jul 06 20:12:20.814091 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814063 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="602cda8d-f8c4-4a6a-9a04-5759fc1af23c" containerName="pull" Jul 06 20:12:20.814091 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814069 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5884ee3f-4633-4425-a422-abcb819e01b5" containerName="extract" Jul 06 20:12:20.814091 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814074 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5884ee3f-4633-4425-a422-abcb819e01b5" containerName="extract" Jul 06 20:12:20.814091 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814080 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6485c000-68a5-4ee0-a899-9925bee9b133" containerName="extract" Jul 06 20:12:20.814091 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814085 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6485c000-68a5-4ee0-a899-9925bee9b133" containerName="extract" Jul 06 20:12:20.814091 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814092 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="602cda8d-f8c4-4a6a-9a04-5759fc1af23c" containerName="extract" Jul 06 20:12:20.814091 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814097 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="602cda8d-f8c4-4a6a-9a04-5759fc1af23c" containerName="extract" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814104 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6485c000-68a5-4ee0-a899-9925bee9b133" containerName="util" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814110 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6485c000-68a5-4ee0-a899-9925bee9b133" containerName="util" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814119 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5884ee3f-4633-4425-a422-abcb819e01b5" containerName="pull" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814124 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5884ee3f-4633-4425-a422-abcb819e01b5" containerName="pull" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814136 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5884ee3f-4633-4425-a422-abcb819e01b5" containerName="util" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814141 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5884ee3f-4633-4425-a422-abcb819e01b5" containerName="util" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814147 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6603e100-ee5e-4c3d-9063-0b638e10b392" containerName="extract" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814152 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6603e100-ee5e-4c3d-9063-0b638e10b392" containerName="extract" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814159 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6485c000-68a5-4ee0-a899-9925bee9b133" containerName="pull" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814164 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6485c000-68a5-4ee0-a899-9925bee9b133" containerName="pull" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814174 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6603e100-ee5e-4c3d-9063-0b638e10b392" containerName="pull" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814178 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6603e100-ee5e-4c3d-9063-0b638e10b392" containerName="pull" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814226 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6485c000-68a5-4ee0-a899-9925bee9b133" containerName="extract" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814233 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5884ee3f-4633-4425-a422-abcb819e01b5" containerName="extract" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814241 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="602cda8d-f8c4-4a6a-9a04-5759fc1af23c" containerName="extract" Jul 06 20:12:20.814433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.814246 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6603e100-ee5e-4c3d-9063-0b638e10b392" containerName="extract" Jul 06 20:12:20.818399 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.818380 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:20.831969 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.831942 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch"] Jul 06 20:12:20.845788 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.845769 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-j7rq6\"" Jul 06 20:12:20.937150 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:20.937128 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6dnl\" (UniqueName: \"kubernetes.io/projected/b6561fd7-da94-4c8f-b71b-8e09698875ea-kube-api-access-l6dnl\") pod \"limitador-operator-controller-manager-859d7b78bd-v8qch\" (UID: \"b6561fd7-da94-4c8f-b71b-8e09698875ea\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:21.037466 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:21.037440 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6dnl\" (UniqueName: \"kubernetes.io/projected/b6561fd7-da94-4c8f-b71b-8e09698875ea-kube-api-access-l6dnl\") pod \"limitador-operator-controller-manager-859d7b78bd-v8qch\" (UID: \"b6561fd7-da94-4c8f-b71b-8e09698875ea\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:21.064077 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:21.064020 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6dnl\" (UniqueName: \"kubernetes.io/projected/b6561fd7-da94-4c8f-b71b-8e09698875ea-kube-api-access-l6dnl\") pod \"limitador-operator-controller-manager-859d7b78bd-v8qch\" (UID: \"b6561fd7-da94-4c8f-b71b-8e09698875ea\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:21.129492 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:21.129461 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:21.276925 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:12:21.276895 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6561fd7_da94_4c8f_b71b_8e09698875ea.slice/crio-a3c6727fa3efd284fbc3a461ba3b9bc2c50c9854bfa9fed680325c6291a6190d WatchSource:0}: Error finding container a3c6727fa3efd284fbc3a461ba3b9bc2c50c9854bfa9fed680325c6291a6190d: Status 404 returned error can't find the container with id a3c6727fa3efd284fbc3a461ba3b9bc2c50c9854bfa9fed680325c6291a6190d Jul 06 20:12:21.280979 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:21.280958 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch"] Jul 06 20:12:21.572832 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:21.572800 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" event={"ID":"b6561fd7-da94-4c8f-b71b-8e09698875ea","Type":"ContainerStarted","Data":"a3c6727fa3efd284fbc3a461ba3b9bc2c50c9854bfa9fed680325c6291a6190d"} Jul 06 20:12:24.593199 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:24.593163 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" event={"ID":"b6561fd7-da94-4c8f-b71b-8e09698875ea","Type":"ContainerStarted","Data":"e7575fe1c0b5ffd5e18e753e29556841c5232fc0f7057c0f0000e08e1cfac6c9"} Jul 06 20:12:24.593554 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:24.593263 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:24.614860 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:24.614818 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" podStartSLOduration=1.819334059 podStartE2EDuration="4.614805006s" totalImagesPullingTime="2.795470947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:12:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:12:24.612001481 +0000 UTC m=+397.738913586" watchObservedRunningTime="2026-07-06 20:12:24.614805006 +0000 UTC m=+397.741717110" Jul 06 20:12:26.015513 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:26.015479 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54"] Jul 06 20:12:26.019062 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:26.019044 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:26.021608 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:26.021592 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-pn8tl\"" Jul 06 20:12:26.035011 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:26.034990 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54"] Jul 06 20:12:26.079961 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:26.079931 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9thzk\" (UniqueName: \"kubernetes.io/projected/123de1f0-7371-46e9-a82d-83a5b424797e-kube-api-access-9thzk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" (UID: \"123de1f0-7371-46e9-a82d-83a5b424797e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:26.080057 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:26.080002 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/123de1f0-7371-46e9-a82d-83a5b424797e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" (UID: \"123de1f0-7371-46e9-a82d-83a5b424797e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:26.181115 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:26.181091 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/123de1f0-7371-46e9-a82d-83a5b424797e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" (UID: \"123de1f0-7371-46e9-a82d-83a5b424797e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:26.181242 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:26.181167 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9thzk\" (UniqueName: \"kubernetes.io/projected/123de1f0-7371-46e9-a82d-83a5b424797e-kube-api-access-9thzk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" (UID: \"123de1f0-7371-46e9-a82d-83a5b424797e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:26.181482 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:26.181460 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/123de1f0-7371-46e9-a82d-83a5b424797e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" (UID: \"123de1f0-7371-46e9-a82d-83a5b424797e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:26.199045 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:26.199018 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9thzk\" (UniqueName: \"kubernetes.io/projected/123de1f0-7371-46e9-a82d-83a5b424797e-kube-api-access-9thzk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" (UID: \"123de1f0-7371-46e9-a82d-83a5b424797e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:26.328858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:26.328799 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:26.473804 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:26.473769 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54"] Jul 06 20:12:26.478137 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:12:26.478102 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod123de1f0_7371_46e9_a82d_83a5b424797e.slice/crio-02417e09c48166fc666aa83d971b2a9d21555b1507b880234be6cbe4f89aafbd WatchSource:0}: Error finding container 02417e09c48166fc666aa83d971b2a9d21555b1507b880234be6cbe4f89aafbd: Status 404 returned error can't find the container with id 02417e09c48166fc666aa83d971b2a9d21555b1507b880234be6cbe4f89aafbd Jul 06 20:12:26.604777 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:26.604710 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" event={"ID":"123de1f0-7371-46e9-a82d-83a5b424797e","Type":"ContainerStarted","Data":"02417e09c48166fc666aa83d971b2a9d21555b1507b880234be6cbe4f89aafbd"} Jul 06 20:12:28.706983 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:28.706951 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk"] Jul 06 20:12:28.710539 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:28.710511 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk" Jul 06 20:12:28.713824 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:28.713798 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-b574b\"" Jul 06 20:12:28.713985 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:28.713843 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 20:12:28.723111 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:28.723089 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk"] Jul 06 20:12:28.802773 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:28.802740 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58k5t\" (UniqueName: \"kubernetes.io/projected/0732e11e-ca32-46dc-b9fe-a7ad6ef72eae-kube-api-access-58k5t\") pod \"dns-operator-controller-manager-864dd78c4c-drtjk\" (UID: \"0732e11e-ca32-46dc-b9fe-a7ad6ef72eae\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk" Jul 06 20:12:28.904010 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:28.903984 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58k5t\" (UniqueName: \"kubernetes.io/projected/0732e11e-ca32-46dc-b9fe-a7ad6ef72eae-kube-api-access-58k5t\") pod \"dns-operator-controller-manager-864dd78c4c-drtjk\" (UID: \"0732e11e-ca32-46dc-b9fe-a7ad6ef72eae\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk" Jul 06 20:12:28.917103 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:28.917084 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58k5t\" (UniqueName: \"kubernetes.io/projected/0732e11e-ca32-46dc-b9fe-a7ad6ef72eae-kube-api-access-58k5t\") pod \"dns-operator-controller-manager-864dd78c4c-drtjk\" (UID: \"0732e11e-ca32-46dc-b9fe-a7ad6ef72eae\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk" Jul 06 20:12:29.024841 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:29.024814 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk" Jul 06 20:12:29.181141 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:29.181114 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk"] Jul 06 20:12:29.183128 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:12:29.183105 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0732e11e_ca32_46dc_b9fe_a7ad6ef72eae.slice/crio-61776b2d2f7a5b562bc300d6f4e10b3d8ba5dbab49b96989717abeb42e2b128b WatchSource:0}: Error finding container 61776b2d2f7a5b562bc300d6f4e10b3d8ba5dbab49b96989717abeb42e2b128b: Status 404 returned error can't find the container with id 61776b2d2f7a5b562bc300d6f4e10b3d8ba5dbab49b96989717abeb42e2b128b Jul 06 20:12:29.626118 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:29.626066 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk" event={"ID":"0732e11e-ca32-46dc-b9fe-a7ad6ef72eae","Type":"ContainerStarted","Data":"61776b2d2f7a5b562bc300d6f4e10b3d8ba5dbab49b96989717abeb42e2b128b"} Jul 06 20:12:31.639247 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:31.639163 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk" event={"ID":"0732e11e-ca32-46dc-b9fe-a7ad6ef72eae","Type":"ContainerStarted","Data":"029fe1b84440bf71443f633f66ff7d3a5c303ef91d64bebc378485007a1b264f"} Jul 06 20:12:31.639707 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:31.639296 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk" Jul 06 20:12:31.664183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:31.664130 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk" podStartSLOduration=1.521985983 podStartE2EDuration="3.664118129s" totalImagesPullingTime="2.142132146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:12:31.662554391 +0000 UTC m=+404.789466497" watchObservedRunningTime="2026-07-06 20:12:31.664118129 +0000 UTC m=+404.791030233" Jul 06 20:12:32.264243 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:32.264202 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6"] Jul 06 20:12:32.268539 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:32.268517 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" Jul 06 20:12:32.273992 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:32.273971 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-v5nxb\"" Jul 06 20:12:32.292234 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:32.292210 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6"] Jul 06 20:12:32.333607 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:32.333578 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbspj\" (UniqueName: \"kubernetes.io/projected/ba75fdd1-3a5a-46cb-a655-a46013cf495e-kube-api-access-jbspj\") pod \"authorino-operator-75cc8d5cb7-tj6k6\" (UID: \"ba75fdd1-3a5a-46cb-a655-a46013cf495e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" Jul 06 20:12:32.434180 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:32.434146 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbspj\" (UniqueName: \"kubernetes.io/projected/ba75fdd1-3a5a-46cb-a655-a46013cf495e-kube-api-access-jbspj\") pod \"authorino-operator-75cc8d5cb7-tj6k6\" (UID: \"ba75fdd1-3a5a-46cb-a655-a46013cf495e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" Jul 06 20:12:32.444731 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:32.444704 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbspj\" (UniqueName: \"kubernetes.io/projected/ba75fdd1-3a5a-46cb-a655-a46013cf495e-kube-api-access-jbspj\") pod \"authorino-operator-75cc8d5cb7-tj6k6\" (UID: \"ba75fdd1-3a5a-46cb-a655-a46013cf495e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" Jul 06 20:12:32.580519 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:32.580433 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" Jul 06 20:12:33.149278 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:33.149248 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6"] Jul 06 20:12:33.150031 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:12:33.150011 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba75fdd1_3a5a_46cb_a655_a46013cf495e.slice/crio-eeb81287938e4cc25c43b3b8621a13a5cd03df2deac36878fdb530fdda2fc4aa WatchSource:0}: Error finding container eeb81287938e4cc25c43b3b8621a13a5cd03df2deac36878fdb530fdda2fc4aa: Status 404 returned error can't find the container with id eeb81287938e4cc25c43b3b8621a13a5cd03df2deac36878fdb530fdda2fc4aa Jul 06 20:12:33.655594 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:33.655556 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" event={"ID":"123de1f0-7371-46e9-a82d-83a5b424797e","Type":"ContainerStarted","Data":"2243f733347ec7c5a88e2b71bd0bbd381f6ba0c56eeec3f4f5b8d4dfef872f7b"} Jul 06 20:12:33.655944 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:33.655669 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:33.657172 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:33.657142 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" event={"ID":"ba75fdd1-3a5a-46cb-a655-a46013cf495e","Type":"ContainerStarted","Data":"eeb81287938e4cc25c43b3b8621a13a5cd03df2deac36878fdb530fdda2fc4aa"} Jul 06 20:12:33.681103 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:33.681055 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" podStartSLOduration=2.089381451 podStartE2EDuration="8.681042137s" totalImagesPullingTime="6.591660686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:12:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:12:33.679739101 +0000 UTC m=+406.806651209" watchObservedRunningTime="2026-07-06 20:12:33.681042137 +0000 UTC m=+406.807954241" Jul 06 20:12:35.600900 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:35.600864 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:35.671596 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:35.671565 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" event={"ID":"ba75fdd1-3a5a-46cb-a655-a46013cf495e","Type":"ContainerStarted","Data":"c91c4ebe9375454a74bcba256ead184e4d2b92ca8572215e5e026fb01df87266"} Jul 06 20:12:35.671757 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:35.671689 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" Jul 06 20:12:35.701333 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:35.701284 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" podStartSLOduration=1.329751933 podStartE2EDuration="3.70127007s" totalImagesPullingTime="2.371518137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:12:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:12:35.699626277 +0000 UTC m=+408.826538381" watchObservedRunningTime="2026-07-06 20:12:35.70127007 +0000 UTC m=+408.828182165" Jul 06 20:12:36.137156 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.137124 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27"] Jul 06 20:12:36.140773 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.140753 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" Jul 06 20:12:36.143412 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.143378 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 06 20:12:36.143521 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.143384 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fgg7s\"" Jul 06 20:12:36.143571 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.143558 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 06 20:12:36.155645 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.155615 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27"] Jul 06 20:12:36.271674 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.271646 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7e2f1dd2-f940-4c42-89f3-e8691ec33679-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-b5j27\" (UID: \"7e2f1dd2-f940-4c42-89f3-e8691ec33679\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" Jul 06 20:12:36.271845 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.271695 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e2f1dd2-f940-4c42-89f3-e8691ec33679-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-b5j27\" (UID: \"7e2f1dd2-f940-4c42-89f3-e8691ec33679\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" Jul 06 20:12:36.271845 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.271771 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thvht\" (UniqueName: \"kubernetes.io/projected/7e2f1dd2-f940-4c42-89f3-e8691ec33679-kube-api-access-thvht\") pod \"kuadrant-console-plugin-66b5774985-b5j27\" (UID: \"7e2f1dd2-f940-4c42-89f3-e8691ec33679\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" Jul 06 20:12:36.373081 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.373055 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7e2f1dd2-f940-4c42-89f3-e8691ec33679-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-b5j27\" (UID: \"7e2f1dd2-f940-4c42-89f3-e8691ec33679\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" Jul 06 20:12:36.373217 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.373091 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e2f1dd2-f940-4c42-89f3-e8691ec33679-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-b5j27\" (UID: \"7e2f1dd2-f940-4c42-89f3-e8691ec33679\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" Jul 06 20:12:36.373217 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.373128 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thvht\" (UniqueName: \"kubernetes.io/projected/7e2f1dd2-f940-4c42-89f3-e8691ec33679-kube-api-access-thvht\") pod \"kuadrant-console-plugin-66b5774985-b5j27\" (UID: \"7e2f1dd2-f940-4c42-89f3-e8691ec33679\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" Jul 06 20:12:36.373329 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:36.373213 2583 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 06 20:12:36.373329 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:36.373283 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e2f1dd2-f940-4c42-89f3-e8691ec33679-plugin-serving-cert podName:7e2f1dd2-f940-4c42-89f3-e8691ec33679 nodeName:}" failed. No retries permitted until 2026-07-06 20:12:36.873260658 +0000 UTC m=+410.000172754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/7e2f1dd2-f940-4c42-89f3-e8691ec33679-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-b5j27" (UID: "7e2f1dd2-f940-4c42-89f3-e8691ec33679") : secret "plugin-serving-cert" not found Jul 06 20:12:36.373741 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.373717 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7e2f1dd2-f940-4c42-89f3-e8691ec33679-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-b5j27\" (UID: \"7e2f1dd2-f940-4c42-89f3-e8691ec33679\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" Jul 06 20:12:36.382936 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.382915 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thvht\" (UniqueName: \"kubernetes.io/projected/7e2f1dd2-f940-4c42-89f3-e8691ec33679-kube-api-access-thvht\") pod \"kuadrant-console-plugin-66b5774985-b5j27\" (UID: \"7e2f1dd2-f940-4c42-89f3-e8691ec33679\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" Jul 06 20:12:36.876936 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.876903 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e2f1dd2-f940-4c42-89f3-e8691ec33679-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-b5j27\" (UID: \"7e2f1dd2-f940-4c42-89f3-e8691ec33679\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" Jul 06 20:12:36.879064 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:36.879046 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e2f1dd2-f940-4c42-89f3-e8691ec33679-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-b5j27\" (UID: \"7e2f1dd2-f940-4c42-89f3-e8691ec33679\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" Jul 06 20:12:37.050090 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.050061 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" Jul 06 20:12:37.123496 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.123462 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cbcdfd7f9-qh25x"] Jul 06 20:12:37.129270 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.129202 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.143831 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.143768 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cbcdfd7f9-qh25x"] Jul 06 20:12:37.210179 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.210153 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27"] Jul 06 20:12:37.211728 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:12:37.211706 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e2f1dd2_f940_4c42_89f3_e8691ec33679.slice/crio-64d244231a554d7d6a27be62fcdfd8db57324c03d8c949504951d3ac115c4b4c WatchSource:0}: Error finding container 64d244231a554d7d6a27be62fcdfd8db57324c03d8c949504951d3ac115c4b4c: Status 404 returned error can't find the container with id 64d244231a554d7d6a27be62fcdfd8db57324c03d8c949504951d3ac115c4b4c Jul 06 20:12:37.280747 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.280723 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33f8fb7e-4604-40f9-a874-6154912fcbaf-oauth-serving-cert\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.280901 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.280758 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33f8fb7e-4604-40f9-a874-6154912fcbaf-console-config\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.280901 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.280839 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33f8fb7e-4604-40f9-a874-6154912fcbaf-service-ca\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.281035 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.280908 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33f8fb7e-4604-40f9-a874-6154912fcbaf-trusted-ca-bundle\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.281035 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.280972 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33f8fb7e-4604-40f9-a874-6154912fcbaf-console-serving-cert\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.281035 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.281016 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33f8fb7e-4604-40f9-a874-6154912fcbaf-console-oauth-config\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.281169 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.281045 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns9qj\" (UniqueName: \"kubernetes.io/projected/33f8fb7e-4604-40f9-a874-6154912fcbaf-kube-api-access-ns9qj\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.381456 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.381393 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33f8fb7e-4604-40f9-a874-6154912fcbaf-service-ca\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.381456 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.381423 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33f8fb7e-4604-40f9-a874-6154912fcbaf-trusted-ca-bundle\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.381456 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.381444 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33f8fb7e-4604-40f9-a874-6154912fcbaf-console-serving-cert\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.381633 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.381465 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33f8fb7e-4604-40f9-a874-6154912fcbaf-console-oauth-config\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.381633 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.381490 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns9qj\" (UniqueName: \"kubernetes.io/projected/33f8fb7e-4604-40f9-a874-6154912fcbaf-kube-api-access-ns9qj\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.381633 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.381537 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33f8fb7e-4604-40f9-a874-6154912fcbaf-oauth-serving-cert\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.381633 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.381563 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33f8fb7e-4604-40f9-a874-6154912fcbaf-console-config\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.382294 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.382262 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33f8fb7e-4604-40f9-a874-6154912fcbaf-service-ca\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.382416 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.382324 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33f8fb7e-4604-40f9-a874-6154912fcbaf-console-config\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.382416 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.382337 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33f8fb7e-4604-40f9-a874-6154912fcbaf-trusted-ca-bundle\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.382416 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.382381 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33f8fb7e-4604-40f9-a874-6154912fcbaf-oauth-serving-cert\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.383969 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.383951 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33f8fb7e-4604-40f9-a874-6154912fcbaf-console-serving-cert\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.384120 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.384105 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33f8fb7e-4604-40f9-a874-6154912fcbaf-console-oauth-config\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.392399 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.392380 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns9qj\" (UniqueName: \"kubernetes.io/projected/33f8fb7e-4604-40f9-a874-6154912fcbaf-kube-api-access-ns9qj\") pod \"console-7cbcdfd7f9-qh25x\" (UID: \"33f8fb7e-4604-40f9-a874-6154912fcbaf\") " pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.439537 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.439517 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:37.580179 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.580153 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cbcdfd7f9-qh25x"] Jul 06 20:12:37.581812 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:12:37.581786 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33f8fb7e_4604_40f9_a874_6154912fcbaf.slice/crio-6c2195aebeba73e4ec0cdf0547598bb4a94de1663cb8c5c4a1c1906a4c0e52f7 WatchSource:0}: Error finding container 6c2195aebeba73e4ec0cdf0547598bb4a94de1663cb8c5c4a1c1906a4c0e52f7: Status 404 returned error can't find the container with id 6c2195aebeba73e4ec0cdf0547598bb4a94de1663cb8c5c4a1c1906a4c0e52f7 Jul 06 20:12:37.684696 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.684667 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbcdfd7f9-qh25x" event={"ID":"33f8fb7e-4604-40f9-a874-6154912fcbaf","Type":"ContainerStarted","Data":"43cc6e93f181b7c1296af8cf50d26368171e7c86de69588a364cd43f04e2c658"} Jul 06 20:12:37.684863 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.684718 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbcdfd7f9-qh25x" event={"ID":"33f8fb7e-4604-40f9-a874-6154912fcbaf","Type":"ContainerStarted","Data":"6c2195aebeba73e4ec0cdf0547598bb4a94de1663cb8c5c4a1c1906a4c0e52f7"} Jul 06 20:12:37.686078 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.686050 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" event={"ID":"7e2f1dd2-f940-4c42-89f3-e8691ec33679","Type":"ContainerStarted","Data":"64d244231a554d7d6a27be62fcdfd8db57324c03d8c949504951d3ac115c4b4c"} Jul 06 20:12:37.710068 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:37.710024 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cbcdfd7f9-qh25x" podStartSLOduration=0.710010753 podStartE2EDuration="710.010753ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:12:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:12:37.707505521 +0000 UTC m=+410.834417627" watchObservedRunningTime="2026-07-06 20:12:37.710010753 +0000 UTC m=+410.836922856" Jul 06 20:12:42.649028 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:42.648994 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk" Jul 06 20:12:44.665617 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:44.665582 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:46.226075 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.226039 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg"] Jul 06 20:12:46.230075 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.230053 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:46.249910 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.249885 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg"] Jul 06 20:12:46.337079 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.337048 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg"] Jul 06 20:12:46.337362 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.337341 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-l7ppg], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" podUID="5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3" Jul 06 20:12:46.353897 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.353857 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg"] Jul 06 20:12:46.360787 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.360745 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-s8fbg\" (UID: \"5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:46.360943 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.360918 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7ppg\" (UniqueName: \"kubernetes.io/projected/5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3-kube-api-access-l7ppg\") pod \"kuadrant-operator-controller-manager-55467d9f7-s8fbg\" (UID: \"5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:46.368483 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.368460 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54"] Jul 06 20:12:46.368674 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.368650 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" containerName="manager" containerID="cri-o://2243f733347ec7c5a88e2b71bd0bbd381f6ba0c56eeec3f4f5b8d4dfef872f7b" gracePeriod=2 Jul 06 20:12:46.379827 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.379795 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54"] Jul 06 20:12:46.441518 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.441488 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch"] Jul 06 20:12:46.441933 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.441865 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" containerName="manager" containerID="cri-o://e7575fe1c0b5ffd5e18e753e29556841c5232fc0f7057c0f0000e08e1cfac6c9" gracePeriod=2 Jul 06 20:12:46.444191 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.444162 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:46.450687 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.450665 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp"] Jul 06 20:12:46.451571 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.451251 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" containerName="manager" Jul 06 20:12:46.451571 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.451274 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" containerName="manager" Jul 06 20:12:46.451571 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.451389 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" containerName="manager" Jul 06 20:12:46.454813 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.454793 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch"] Jul 06 20:12:46.455087 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.454925 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" Jul 06 20:12:46.461862 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.461838 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-s8fbg\" (UID: \"5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:46.461997 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.461951 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7ppg\" (UniqueName: \"kubernetes.io/projected/5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3-kube-api-access-l7ppg\") pod \"kuadrant-operator-controller-manager-55467d9f7-s8fbg\" (UID: \"5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:46.462242 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.462221 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-s8fbg\" (UID: \"5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:46.470731 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.470681 2583 projected.go:196] Error preparing data for projected volume kube-api-access-l7ppg for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg: failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-s8fbg" not found Jul 06 20:12:46.470836 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.470747 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3-kube-api-access-l7ppg podName:5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3 nodeName:}" failed. No retries permitted until 2026-07-06 20:12:46.970726567 +0000 UTC m=+420.097638670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l7ppg" (UniqueName: "kubernetes.io/projected/5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3-kube-api-access-l7ppg") pod "kuadrant-operator-controller-manager-55467d9f7-s8fbg" (UID: "5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3") : failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-s8fbg" not found Jul 06 20:12:46.474413 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.474388 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml"] Jul 06 20:12:46.474982 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.474961 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" containerName="manager" Jul 06 20:12:46.474982 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.474984 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" containerName="manager" Jul 06 20:12:46.475139 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.475087 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" containerName="manager" Jul 06 20:12:46.478658 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.478589 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp"] Jul 06 20:12:46.478764 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.478694 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" Jul 06 20:12:46.483036 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.483014 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg"] Jul 06 20:12:46.486567 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.486550 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg" Jul 06 20:12:46.498013 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.497989 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml"] Jul 06 20:12:46.511246 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.511120 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v8qch\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:46.513054 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.512947 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:46.517718 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.517670 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg"] Jul 06 20:12:46.545172 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.545141 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v8qch\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:46.562815 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.562701 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d5676ab-0efc-40b0-9250-e73143cb417d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zwrml\" (UID: \"6d5676ab-0efc-40b0-9250-e73143cb417d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" Jul 06 20:12:46.562815 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.562740 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkqwh\" (UniqueName: \"kubernetes.io/projected/fb33694d-e0bd-479a-89cc-1cdc2be67151-kube-api-access-hkqwh\") pod \"limitador-operator-controller-manager-859d7b78bd-2k7bg\" (UID: \"fb33694d-e0bd-479a-89cc-1cdc2be67151\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg" Jul 06 20:12:46.562988 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.562846 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/830a129b-9191-4abc-b127-cb30c7ff8aa7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hzdpp\" (UID: \"830a129b-9191-4abc-b127-cb30c7ff8aa7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" Jul 06 20:12:46.562988 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.562898 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9fs9\" (UniqueName: \"kubernetes.io/projected/830a129b-9191-4abc-b127-cb30c7ff8aa7-kube-api-access-k9fs9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hzdpp\" (UID: \"830a129b-9191-4abc-b127-cb30c7ff8aa7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" Jul 06 20:12:46.562988 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.562925 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcjlh\" (UniqueName: \"kubernetes.io/projected/6d5676ab-0efc-40b0-9250-e73143cb417d-kube-api-access-gcjlh\") pod \"kuadrant-operator-controller-manager-86878c769f-zwrml\" (UID: \"6d5676ab-0efc-40b0-9250-e73143cb417d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" Jul 06 20:12:46.568093 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.568051 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:46.662925 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.662786 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:46.663364 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.663344 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d5676ab-0efc-40b0-9250-e73143cb417d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zwrml\" (UID: \"6d5676ab-0efc-40b0-9250-e73143cb417d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" Jul 06 20:12:46.663461 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.663378 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkqwh\" (UniqueName: \"kubernetes.io/projected/fb33694d-e0bd-479a-89cc-1cdc2be67151-kube-api-access-hkqwh\") pod \"limitador-operator-controller-manager-859d7b78bd-2k7bg\" (UID: \"fb33694d-e0bd-479a-89cc-1cdc2be67151\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg" Jul 06 20:12:46.663524 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.663479 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/830a129b-9191-4abc-b127-cb30c7ff8aa7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hzdpp\" (UID: \"830a129b-9191-4abc-b127-cb30c7ff8aa7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" Jul 06 20:12:46.663524 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.663519 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9fs9\" (UniqueName: \"kubernetes.io/projected/830a129b-9191-4abc-b127-cb30c7ff8aa7-kube-api-access-k9fs9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hzdpp\" (UID: \"830a129b-9191-4abc-b127-cb30c7ff8aa7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" Jul 06 20:12:46.663640 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.663546 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcjlh\" (UniqueName: \"kubernetes.io/projected/6d5676ab-0efc-40b0-9250-e73143cb417d-kube-api-access-gcjlh\") pod \"kuadrant-operator-controller-manager-86878c769f-zwrml\" (UID: \"6d5676ab-0efc-40b0-9250-e73143cb417d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" Jul 06 20:12:46.663797 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.663770 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d5676ab-0efc-40b0-9250-e73143cb417d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zwrml\" (UID: \"6d5676ab-0efc-40b0-9250-e73143cb417d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" Jul 06 20:12:46.664144 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.664106 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/830a129b-9191-4abc-b127-cb30c7ff8aa7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hzdpp\" (UID: \"830a129b-9191-4abc-b127-cb30c7ff8aa7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" Jul 06 20:12:46.667249 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.667221 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:46.669380 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.669352 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v8qch\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:46.675259 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.675238 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkqwh\" (UniqueName: \"kubernetes.io/projected/fb33694d-e0bd-479a-89cc-1cdc2be67151-kube-api-access-hkqwh\") pod \"limitador-operator-controller-manager-859d7b78bd-2k7bg\" (UID: \"fb33694d-e0bd-479a-89cc-1cdc2be67151\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg" Jul 06 20:12:46.678577 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.678558 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" Jul 06 20:12:46.680285 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.680261 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9fs9\" (UniqueName: \"kubernetes.io/projected/830a129b-9191-4abc-b127-cb30c7ff8aa7-kube-api-access-k9fs9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hzdpp\" (UID: \"830a129b-9191-4abc-b127-cb30c7ff8aa7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" Jul 06 20:12:46.680602 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.680557 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcjlh\" (UniqueName: \"kubernetes.io/projected/6d5676ab-0efc-40b0-9250-e73143cb417d-kube-api-access-gcjlh\") pod \"kuadrant-operator-controller-manager-86878c769f-zwrml\" (UID: \"6d5676ab-0efc-40b0-9250-e73143cb417d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" Jul 06 20:12:46.681101 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.681067 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:46.713963 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.713918 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:46.715092 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.714767 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v8qch\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:46.722207 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.717178 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:46.722207 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.718997 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v8qch\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:46.748672 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.748605 2583 generic.go:356] "Generic (PLEG): container finished" podID="123de1f0-7371-46e9-a82d-83a5b424797e" containerID="2243f733347ec7c5a88e2b71bd0bbd381f6ba0c56eeec3f4f5b8d4dfef872f7b" exitCode=0 Jul 06 20:12:46.748785 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.748729 2583 scope.go:117] "RemoveContainer" containerID="2243f733347ec7c5a88e2b71bd0bbd381f6ba0c56eeec3f4f5b8d4dfef872f7b" Jul 06 20:12:46.748928 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.748910 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:46.750732 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.750705 2583 generic.go:356] "Generic (PLEG): container finished" podID="b6561fd7-da94-4c8f-b71b-8e09698875ea" containerID="e7575fe1c0b5ffd5e18e753e29556841c5232fc0f7057c0f0000e08e1cfac6c9" exitCode=0 Jul 06 20:12:46.750855 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.750773 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:46.750999 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.750947 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:46.751510 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.751483 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v8qch\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:46.753573 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.753490 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:46.755446 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.755421 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:46.757144 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.757071 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v8qch\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:46.758937 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.758910 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-s8fbg\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:46.760060 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.760037 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:46.761284 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.761267 2583 scope.go:117] "RemoveContainer" containerID="2243f733347ec7c5a88e2b71bd0bbd381f6ba0c56eeec3f4f5b8d4dfef872f7b" Jul 06 20:12:46.761651 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.761628 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2243f733347ec7c5a88e2b71bd0bbd381f6ba0c56eeec3f4f5b8d4dfef872f7b\": container with ID starting with 2243f733347ec7c5a88e2b71bd0bbd381f6ba0c56eeec3f4f5b8d4dfef872f7b not found: ID does not exist" containerID="2243f733347ec7c5a88e2b71bd0bbd381f6ba0c56eeec3f4f5b8d4dfef872f7b" Jul 06 20:12:46.761752 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.761661 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2243f733347ec7c5a88e2b71bd0bbd381f6ba0c56eeec3f4f5b8d4dfef872f7b"} err="failed to get container status \"2243f733347ec7c5a88e2b71bd0bbd381f6ba0c56eeec3f4f5b8d4dfef872f7b\": rpc error: code = NotFound desc = could not find container \"2243f733347ec7c5a88e2b71bd0bbd381f6ba0c56eeec3f4f5b8d4dfef872f7b\": container with ID starting with 2243f733347ec7c5a88e2b71bd0bbd381f6ba0c56eeec3f4f5b8d4dfef872f7b not found: ID does not exist" Jul 06 20:12:46.761752 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.761687 2583 scope.go:117] "RemoveContainer" containerID="e7575fe1c0b5ffd5e18e753e29556841c5232fc0f7057c0f0000e08e1cfac6c9" Jul 06 20:12:46.762262 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.762011 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:46.764025 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.763996 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v8qch\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:46.764396 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.764375 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9thzk\" (UniqueName: \"kubernetes.io/projected/123de1f0-7371-46e9-a82d-83a5b424797e-kube-api-access-9thzk\") pod \"123de1f0-7371-46e9-a82d-83a5b424797e\" (UID: \"123de1f0-7371-46e9-a82d-83a5b424797e\") " Jul 06 20:12:46.764497 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.764430 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/123de1f0-7371-46e9-a82d-83a5b424797e-extensions-socket-volume\") pod \"123de1f0-7371-46e9-a82d-83a5b424797e\" (UID: \"123de1f0-7371-46e9-a82d-83a5b424797e\") " Jul 06 20:12:46.765223 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.765200 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/123de1f0-7371-46e9-a82d-83a5b424797e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "123de1f0-7371-46e9-a82d-83a5b424797e" (UID: "123de1f0-7371-46e9-a82d-83a5b424797e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:12:46.765795 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.765757 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-s8fbg\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:46.767117 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.767092 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/123de1f0-7371-46e9-a82d-83a5b424797e-kube-api-access-9thzk" (OuterVolumeSpecName: "kube-api-access-9thzk") pod "123de1f0-7371-46e9-a82d-83a5b424797e" (UID: "123de1f0-7371-46e9-a82d-83a5b424797e"). InnerVolumeSpecName "kube-api-access-9thzk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:12:46.773178 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.773145 2583 scope.go:117] "RemoveContainer" containerID="e7575fe1c0b5ffd5e18e753e29556841c5232fc0f7057c0f0000e08e1cfac6c9" Jul 06 20:12:46.773469 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:46.773440 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7575fe1c0b5ffd5e18e753e29556841c5232fc0f7057c0f0000e08e1cfac6c9\": container with ID starting with e7575fe1c0b5ffd5e18e753e29556841c5232fc0f7057c0f0000e08e1cfac6c9 not found: ID does not exist" containerID="e7575fe1c0b5ffd5e18e753e29556841c5232fc0f7057c0f0000e08e1cfac6c9" Jul 06 20:12:46.773551 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.773478 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7575fe1c0b5ffd5e18e753e29556841c5232fc0f7057c0f0000e08e1cfac6c9"} err="failed to get container status \"e7575fe1c0b5ffd5e18e753e29556841c5232fc0f7057c0f0000e08e1cfac6c9\": rpc error: code = NotFound desc = could not find container \"e7575fe1c0b5ffd5e18e753e29556841c5232fc0f7057c0f0000e08e1cfac6c9\": container with ID starting with e7575fe1c0b5ffd5e18e753e29556841c5232fc0f7057c0f0000e08e1cfac6c9 not found: ID does not exist" Jul 06 20:12:46.826368 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.826349 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" Jul 06 20:12:46.834472 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.834427 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" Jul 06 20:12:46.844301 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.844275 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg" Jul 06 20:12:46.865640 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.865492 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6dnl\" (UniqueName: \"kubernetes.io/projected/b6561fd7-da94-4c8f-b71b-8e09698875ea-kube-api-access-l6dnl\") pod \"b6561fd7-da94-4c8f-b71b-8e09698875ea\" (UID: \"b6561fd7-da94-4c8f-b71b-8e09698875ea\") " Jul 06 20:12:46.865640 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.865541 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3-extensions-socket-volume\") pod \"5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3\" (UID: \"5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3\") " Jul 06 20:12:46.865917 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.865893 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9thzk\" (UniqueName: \"kubernetes.io/projected/123de1f0-7371-46e9-a82d-83a5b424797e-kube-api-access-9thzk\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:46.866007 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.865922 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7ppg\" (UniqueName: \"kubernetes.io/projected/5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3-kube-api-access-l7ppg\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:46.866007 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.865925 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3" (UID: "5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:12:46.866007 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.865938 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/123de1f0-7371-46e9-a82d-83a5b424797e-extensions-socket-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:46.868229 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.868198 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6561fd7-da94-4c8f-b71b-8e09698875ea-kube-api-access-l6dnl" (OuterVolumeSpecName: "kube-api-access-l6dnl") pod "b6561fd7-da94-4c8f-b71b-8e09698875ea" (UID: "b6561fd7-da94-4c8f-b71b-8e09698875ea"). InnerVolumeSpecName "kube-api-access-l6dnl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:12:46.967173 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.967073 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6dnl\" (UniqueName: \"kubernetes.io/projected/b6561fd7-da94-4c8f-b71b-8e09698875ea-kube-api-access-l6dnl\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:46.967173 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:46.967117 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3-extensions-socket-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:12:47.051263 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.051239 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp"] Jul 06 20:12:47.053244 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:12:47.053213 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod830a129b_9191_4abc_b127_cb30c7ff8aa7.slice/crio-f0376c7f5dd65b25be43d5233102fbe9a0a600aeeb6b58a5fa0637778b54d389 WatchSource:0}: Error finding container f0376c7f5dd65b25be43d5233102fbe9a0a600aeeb6b58a5fa0637778b54d389: Status 404 returned error can't find the container with id f0376c7f5dd65b25be43d5233102fbe9a0a600aeeb6b58a5fa0637778b54d389 Jul 06 20:12:47.064266 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:47.064233 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:47.067834 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:47.067802 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v8qch\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:47.070977 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:47.070948 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-s8fbg\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:47.073462 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:47.073439 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:47.075509 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:47.075489 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v8qch\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:47.078651 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:47.078430 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-s8fbg\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:47.297661 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.297625 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg"] Jul 06 20:12:47.299857 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:12:47.299801 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb33694d_e0bd_479a_89cc_1cdc2be67151.slice/crio-1fe5523738d02f1151c9d7711113b5686a8bbe8dc91ba8bc490c4549a4d1e0d1 WatchSource:0}: Error finding container 1fe5523738d02f1151c9d7711113b5686a8bbe8dc91ba8bc490c4549a4d1e0d1: Status 404 returned error can't find the container with id 1fe5523738d02f1151c9d7711113b5686a8bbe8dc91ba8bc490c4549a4d1e0d1 Jul 06 20:12:47.300714 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:12:47.300690 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d5676ab_0efc_40b0_9250_e73143cb417d.slice/crio-15e08a0e821bcdfdcc3832a93958f96928701399de95ea5124c6622abd1b7e46 WatchSource:0}: Error finding container 15e08a0e821bcdfdcc3832a93958f96928701399de95ea5124c6622abd1b7e46: Status 404 returned error can't find the container with id 15e08a0e821bcdfdcc3832a93958f96928701399de95ea5124c6622abd1b7e46 Jul 06 20:12:47.300817 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.300775 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml"] Jul 06 20:12:47.440371 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.440335 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:47.440720 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.440698 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:47.446437 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.446418 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:47.449258 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:47.449228 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-tdw54\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tdw54" Jul 06 20:12:47.451543 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:47.451514 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v8qch\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v8qch" Jul 06 20:12:47.453738 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:47.453718 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-s8fbg\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:47.485880 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.485843 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="123de1f0-7371-46e9-a82d-83a5b424797e" path="/var/lib/kubelet/pods/123de1f0-7371-46e9-a82d-83a5b424797e/volumes" Jul 06 20:12:47.486347 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.486317 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3" path="/var/lib/kubelet/pods/5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3/volumes" Jul 06 20:12:47.486614 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.486598 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6561fd7-da94-4c8f-b71b-8e09698875ea" path="/var/lib/kubelet/pods/b6561fd7-da94-4c8f-b71b-8e09698875ea/volumes" Jul 06 20:12:47.757998 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.757965 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" event={"ID":"830a129b-9191-4abc-b127-cb30c7ff8aa7","Type":"ContainerStarted","Data":"f7f8bc25f30d4c9a7ab6dbd52e6372f718a27409ee078be1ed99d28903a708b5"} Jul 06 20:12:47.758203 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.758031 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" Jul 06 20:12:47.758203 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.758050 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" event={"ID":"830a129b-9191-4abc-b127-cb30c7ff8aa7","Type":"ContainerStarted","Data":"f0376c7f5dd65b25be43d5233102fbe9a0a600aeeb6b58a5fa0637778b54d389"} Jul 06 20:12:47.759628 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.759601 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg" event={"ID":"fb33694d-e0bd-479a-89cc-1cdc2be67151","Type":"ContainerStarted","Data":"029a6d438ef50972f58149d9b04806e12358ec70b17d8336e3fd840ba4926967"} Jul 06 20:12:47.759749 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.759650 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg" Jul 06 20:12:47.759749 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.759672 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg" event={"ID":"fb33694d-e0bd-479a-89cc-1cdc2be67151","Type":"ContainerStarted","Data":"1fe5523738d02f1151c9d7711113b5686a8bbe8dc91ba8bc490c4549a4d1e0d1"} Jul 06 20:12:47.761198 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.761170 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" event={"ID":"6d5676ab-0efc-40b0-9250-e73143cb417d","Type":"ContainerStarted","Data":"787f895c8a7173b6032548f57bd4d967205b6c9034ed3fb44f2fc01440eb93e1"} Jul 06 20:12:47.761198 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.761193 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:47.761367 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.761222 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" event={"ID":"6d5676ab-0efc-40b0-9250-e73143cb417d","Type":"ContainerStarted","Data":"15e08a0e821bcdfdcc3832a93958f96928701399de95ea5124c6622abd1b7e46"} Jul 06 20:12:47.761367 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.761345 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" Jul 06 20:12:47.765073 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.765053 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cbcdfd7f9-qh25x" Jul 06 20:12:47.790509 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.790466 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" podStartSLOduration=1.790454312 podStartE2EDuration="1.790454312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:12:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:12:47.78996774 +0000 UTC m=+420.916879861" watchObservedRunningTime="2026-07-06 20:12:47.790454312 +0000 UTC m=+420.917366416" Jul 06 20:12:47.816526 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:47.816494 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-s8fbg\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:47.816773 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.816736 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg" podStartSLOduration=1.816722398 podStartE2EDuration="1.816722398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:12:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:12:47.814236413 +0000 UTC m=+420.941148518" watchObservedRunningTime="2026-07-06 20:12:47.816722398 +0000 UTC m=+420.943634502" Jul 06 20:12:47.869501 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.869452 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" podStartSLOduration=1.8694354720000002 podStartE2EDuration="1.869435472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:12:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:12:47.868616835 +0000 UTC m=+420.995528940" watchObservedRunningTime="2026-07-06 20:12:47.869435472 +0000 UTC m=+420.996347575" Jul 06 20:12:47.898923 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:47.898832 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b4dd5966-rqgrx"] Jul 06 20:12:47.937282 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:12:47.937240 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-s8fbg\" is forbidden: User \"system:node:ip-10-0-130-133.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-133.ec2.internal' and this object" podUID="5c0d6f46-92e6-4aa6-bbb1-f8b3577e45f3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-s8fbg" Jul 06 20:12:58.769724 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:58.769687 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" Jul 06 20:12:58.770197 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:58.770159 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" Jul 06 20:12:58.770268 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:58.770222 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg" Jul 06 20:12:58.886503 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:58.886470 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp"] Jul 06 20:12:58.886712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:12:58.886680 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" podUID="830a129b-9191-4abc-b127-cb30c7ff8aa7" containerName="manager" containerID="cri-o://f7f8bc25f30d4c9a7ab6dbd52e6372f718a27409ee078be1ed99d28903a708b5" gracePeriod=10 Jul 06 20:13:03.251637 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.251614 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" Jul 06 20:13:03.332825 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.332799 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/830a129b-9191-4abc-b127-cb30c7ff8aa7-extensions-socket-volume\") pod \"830a129b-9191-4abc-b127-cb30c7ff8aa7\" (UID: \"830a129b-9191-4abc-b127-cb30c7ff8aa7\") " Jul 06 20:13:03.332959 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.332908 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9fs9\" (UniqueName: \"kubernetes.io/projected/830a129b-9191-4abc-b127-cb30c7ff8aa7-kube-api-access-k9fs9\") pod \"830a129b-9191-4abc-b127-cb30c7ff8aa7\" (UID: \"830a129b-9191-4abc-b127-cb30c7ff8aa7\") " Jul 06 20:13:03.333148 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.333124 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/830a129b-9191-4abc-b127-cb30c7ff8aa7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "830a129b-9191-4abc-b127-cb30c7ff8aa7" (UID: "830a129b-9191-4abc-b127-cb30c7ff8aa7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:13:03.334735 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.334714 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/830a129b-9191-4abc-b127-cb30c7ff8aa7-kube-api-access-k9fs9" (OuterVolumeSpecName: "kube-api-access-k9fs9") pod "830a129b-9191-4abc-b127-cb30c7ff8aa7" (UID: "830a129b-9191-4abc-b127-cb30c7ff8aa7"). InnerVolumeSpecName "kube-api-access-k9fs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:13:03.434297 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.434244 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9fs9\" (UniqueName: \"kubernetes.io/projected/830a129b-9191-4abc-b127-cb30c7ff8aa7-kube-api-access-k9fs9\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:13:03.434297 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.434264 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/830a129b-9191-4abc-b127-cb30c7ff8aa7-extensions-socket-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:13:03.870404 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.870367 2583 generic.go:356] "Generic (PLEG): container finished" podID="830a129b-9191-4abc-b127-cb30c7ff8aa7" containerID="f7f8bc25f30d4c9a7ab6dbd52e6372f718a27409ee078be1ed99d28903a708b5" exitCode=0 Jul 06 20:13:03.870581 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.870434 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" Jul 06 20:13:03.870581 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.870453 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" event={"ID":"830a129b-9191-4abc-b127-cb30c7ff8aa7","Type":"ContainerDied","Data":"f7f8bc25f30d4c9a7ab6dbd52e6372f718a27409ee078be1ed99d28903a708b5"} Jul 06 20:13:03.870581 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.870507 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp" event={"ID":"830a129b-9191-4abc-b127-cb30c7ff8aa7","Type":"ContainerDied","Data":"f0376c7f5dd65b25be43d5233102fbe9a0a600aeeb6b58a5fa0637778b54d389"} Jul 06 20:13:03.870581 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.870529 2583 scope.go:117] "RemoveContainer" containerID="f7f8bc25f30d4c9a7ab6dbd52e6372f718a27409ee078be1ed99d28903a708b5" Jul 06 20:13:03.872209 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.872180 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" event={"ID":"7e2f1dd2-f940-4c42-89f3-e8691ec33679","Type":"ContainerStarted","Data":"e0f05837e2bc3106210c437d0aaaacccdb295cc1bb82ca6ff21749c8f94a4bfe"} Jul 06 20:13:03.879926 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.879907 2583 scope.go:117] "RemoveContainer" containerID="f7f8bc25f30d4c9a7ab6dbd52e6372f718a27409ee078be1ed99d28903a708b5" Jul 06 20:13:03.880206 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:13:03.880172 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7f8bc25f30d4c9a7ab6dbd52e6372f718a27409ee078be1ed99d28903a708b5\": container with ID starting with f7f8bc25f30d4c9a7ab6dbd52e6372f718a27409ee078be1ed99d28903a708b5 not found: ID does not exist" containerID="f7f8bc25f30d4c9a7ab6dbd52e6372f718a27409ee078be1ed99d28903a708b5" Jul 06 20:13:03.880269 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.880215 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f8bc25f30d4c9a7ab6dbd52e6372f718a27409ee078be1ed99d28903a708b5"} err="failed to get container status \"f7f8bc25f30d4c9a7ab6dbd52e6372f718a27409ee078be1ed99d28903a708b5\": rpc error: code = NotFound desc = could not find container \"f7f8bc25f30d4c9a7ab6dbd52e6372f718a27409ee078be1ed99d28903a708b5\": container with ID starting with f7f8bc25f30d4c9a7ab6dbd52e6372f718a27409ee078be1ed99d28903a708b5 not found: ID does not exist" Jul 06 20:13:03.895131 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.895092 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-b5j27" podStartSLOduration=1.936299845 podStartE2EDuration="27.895079172s" totalImagesPullingTime="25.958779327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:12:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:13:03.894222838 +0000 UTC m=+437.021134944" watchObservedRunningTime="2026-07-06 20:13:03.895079172 +0000 UTC m=+437.021991328" Jul 06 20:13:03.913948 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.913928 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp"] Jul 06 20:13:03.917701 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:03.917677 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hzdpp"] Jul 06 20:13:05.486302 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:05.486269 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="830a129b-9191-4abc-b127-cb30c7ff8aa7" path="/var/lib/kubelet/pods/830a129b-9191-4abc-b127-cb30c7ff8aa7/volumes" Jul 06 20:13:12.932047 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:12.931942 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6b4dd5966-rqgrx" podUID="10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a" containerName="console" containerID="cri-o://39b13291a0e89ff1716ab57626e36c5d2174042edca11a116cd390c4289ae2a9" gracePeriod=15 Jul 06 20:13:13.187046 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.186988 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:13:13.309010 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.308979 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-trusted-ca-bundle\") pod \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " Jul 06 20:13:13.309183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.309017 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-service-ca\") pod \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " Jul 06 20:13:13.309183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.309053 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26vnj\" (UniqueName: \"kubernetes.io/projected/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-kube-api-access-26vnj\") pod \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " Jul 06 20:13:13.309183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.309097 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-serving-cert\") pod \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " Jul 06 20:13:13.309183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.309113 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-oauth-serving-cert\") pod \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " Jul 06 20:13:13.309183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.309142 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-config\") pod \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " Jul 06 20:13:13.309183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.309175 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-oauth-config\") pod \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\" (UID: \"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a\") " Jul 06 20:13:13.309508 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.309454 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-service-ca" (OuterVolumeSpecName: "service-ca") pod "10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a" (UID: "10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:13:13.309555 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.309519 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a" (UID: "10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:13:13.309597 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.309559 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a" (UID: "10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:13:13.309639 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.309605 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-config" (OuterVolumeSpecName: "console-config") pod "10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a" (UID: "10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:13:13.311450 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.311425 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a" (UID: "10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:13:13.311719 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.311698 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a" (UID: "10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:13:13.311808 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.311731 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-kube-api-access-26vnj" (OuterVolumeSpecName: "kube-api-access-26vnj") pod "10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a" (UID: "10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a"). InnerVolumeSpecName "kube-api-access-26vnj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:13:13.410314 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.410280 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-serving-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:13:13.410314 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.410306 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-oauth-serving-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:13:13.410314 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.410315 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-config\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:13:13.410499 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.410324 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-console-oauth-config\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:13:13.410499 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.410332 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-trusted-ca-bundle\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:13:13.410499 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.410340 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:13:13.410499 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.410348 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26vnj\" (UniqueName: \"kubernetes.io/projected/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a-kube-api-access-26vnj\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:13:13.928783 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.928694 2583 generic.go:356] "Generic (PLEG): container finished" podID="10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a" containerID="39b13291a0e89ff1716ab57626e36c5d2174042edca11a116cd390c4289ae2a9" exitCode=0 Jul 06 20:13:13.928783 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.928742 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b4dd5966-rqgrx" Jul 06 20:13:13.929002 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.928785 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b4dd5966-rqgrx" event={"ID":"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a","Type":"ContainerDied","Data":"39b13291a0e89ff1716ab57626e36c5d2174042edca11a116cd390c4289ae2a9"} Jul 06 20:13:13.929002 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.928836 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b4dd5966-rqgrx" event={"ID":"10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a","Type":"ContainerDied","Data":"7e10f1ac9c25db791f0f622b61d5cf1191cfebb7bc1aa15846cee988ab3a5207"} Jul 06 20:13:13.929002 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.928856 2583 scope.go:117] "RemoveContainer" containerID="39b13291a0e89ff1716ab57626e36c5d2174042edca11a116cd390c4289ae2a9" Jul 06 20:13:13.937964 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.937702 2583 scope.go:117] "RemoveContainer" containerID="39b13291a0e89ff1716ab57626e36c5d2174042edca11a116cd390c4289ae2a9" Jul 06 20:13:13.938185 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:13:13.938000 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39b13291a0e89ff1716ab57626e36c5d2174042edca11a116cd390c4289ae2a9\": container with ID starting with 39b13291a0e89ff1716ab57626e36c5d2174042edca11a116cd390c4289ae2a9 not found: ID does not exist" containerID="39b13291a0e89ff1716ab57626e36c5d2174042edca11a116cd390c4289ae2a9" Jul 06 20:13:13.938185 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.938038 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39b13291a0e89ff1716ab57626e36c5d2174042edca11a116cd390c4289ae2a9"} err="failed to get container status \"39b13291a0e89ff1716ab57626e36c5d2174042edca11a116cd390c4289ae2a9\": rpc error: code = NotFound desc = could not find container \"39b13291a0e89ff1716ab57626e36c5d2174042edca11a116cd390c4289ae2a9\": container with ID starting with 39b13291a0e89ff1716ab57626e36c5d2174042edca11a116cd390c4289ae2a9 not found: ID does not exist" Jul 06 20:13:13.958823 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.958798 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b4dd5966-rqgrx"] Jul 06 20:13:13.965092 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:13.965073 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b4dd5966-rqgrx"] Jul 06 20:13:15.446305 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.446268 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s"] Jul 06 20:13:15.446901 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.446866 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="830a129b-9191-4abc-b127-cb30c7ff8aa7" containerName="manager" Jul 06 20:13:15.446944 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.446904 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="830a129b-9191-4abc-b127-cb30c7ff8aa7" containerName="manager" Jul 06 20:13:15.446983 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.446953 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a" containerName="console" Jul 06 20:13:15.446983 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.446962 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a" containerName="console" Jul 06 20:13:15.447063 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.447052 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a" containerName="console" Jul 06 20:13:15.447099 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.447069 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="830a129b-9191-4abc-b127-cb30c7ff8aa7" containerName="manager" Jul 06 20:13:15.463803 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.463780 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.467894 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.467855 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 20:13:15.468282 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.468264 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-8pjdc\"" Jul 06 20:13:15.469480 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.468855 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s"] Jul 06 20:13:15.487405 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.487372 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a" path="/var/lib/kubelet/pods/10f4b0d9-d85d-4bdf-8ffe-fccc93237b2a/volumes" Jul 06 20:13:15.526849 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.526820 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dd007b4-41d0-46a4-ba88-5ee40d497497-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.526983 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.526865 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dd007b4-41d0-46a4-ba88-5ee40d497497-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.526983 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.526912 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzpvd\" (UniqueName: \"kubernetes.io/projected/1dd007b4-41d0-46a4-ba88-5ee40d497497-kube-api-access-gzpvd\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.526983 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.526930 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.527114 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.527014 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dd007b4-41d0-46a4-ba88-5ee40d497497-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.527114 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.527056 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.527114 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.527073 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.527114 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.527099 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.527241 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.527123 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.628070 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.628031 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dd007b4-41d0-46a4-ba88-5ee40d497497-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.628231 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.628083 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dd007b4-41d0-46a4-ba88-5ee40d497497-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.628231 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.628109 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzpvd\" (UniqueName: \"kubernetes.io/projected/1dd007b4-41d0-46a4-ba88-5ee40d497497-kube-api-access-gzpvd\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.628231 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.628134 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.628231 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.628183 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dd007b4-41d0-46a4-ba88-5ee40d497497-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.628231 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.628206 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.628231 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.628226 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.628557 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.628248 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.628557 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.628268 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.628699 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.628671 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.628770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.628699 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.628770 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.628758 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.628905 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.628851 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.629103 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.629081 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dd007b4-41d0-46a4-ba88-5ee40d497497-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.630569 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.630549 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dd007b4-41d0-46a4-ba88-5ee40d497497-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.630663 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.630647 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dd007b4-41d0-46a4-ba88-5ee40d497497-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.640964 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.640943 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dd007b4-41d0-46a4-ba88-5ee40d497497-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.642105 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.642089 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzpvd\" (UniqueName: \"kubernetes.io/projected/1dd007b4-41d0-46a4-ba88-5ee40d497497-kube-api-access-gzpvd\") pod \"maas-default-gateway-openshift-default-56654dcf7-jxm4s\" (UID: \"1dd007b4-41d0-46a4-ba88-5ee40d497497\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.784181 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.784152 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:15.935064 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.935042 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s"] Jul 06 20:13:15.936627 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:13:15.936600 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dd007b4_41d0_46a4_ba88_5ee40d497497.slice/crio-feea3deaff2448666c4343fa0bc39410d280c179a6338da2c059b190c1cb61cd WatchSource:0}: Error finding container feea3deaff2448666c4343fa0bc39410d280c179a6338da2c059b190c1cb61cd: Status 404 returned error can't find the container with id feea3deaff2448666c4343fa0bc39410d280c179a6338da2c059b190c1cb61cd Jul 06 20:13:15.948634 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:15.948601 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" event={"ID":"1dd007b4-41d0-46a4-ba88-5ee40d497497","Type":"ContainerStarted","Data":"feea3deaff2448666c4343fa0bc39410d280c179a6338da2c059b190c1cb61cd"} Jul 06 20:13:18.474055 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:18.474019 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:13:18.474361 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:18.474090 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:13:18.474361 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:18.474126 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:13:18.970439 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:18.970406 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" event={"ID":"1dd007b4-41d0-46a4-ba88-5ee40d497497","Type":"ContainerStarted","Data":"9f6a1b9db3e872a38c7c5815625831ed4a1ef22d98c4a329c8b718e61a2180a1"} Jul 06 20:13:19.008626 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:19.008580 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" podStartSLOduration=1.47334101 podStartE2EDuration="4.008565052s" totalImagesPullingTime="2.535224042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:13:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:13:19.006693956 +0000 UTC m=+452.133606060" watchObservedRunningTime="2026-07-06 20:13:19.008565052 +0000 UTC m=+452.135477155" Jul 06 20:13:19.784771 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:19.784740 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:19.789634 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:19.789609 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:19.976184 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:19.976155 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:19.977043 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:19.977024 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:13:20.769826 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:20.769795 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:13:20.816676 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:20.816647 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:13:20.816676 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:20.816679 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:13:20.817095 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:20.816745 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-cdjxm" Jul 06 20:13:20.819389 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:20.819366 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 20:13:20.874021 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:20.873986 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6c277f91-c521-4062-bb13-d23bdeb073aa-config-file\") pod \"limitador-limitador-54d6d988d5-cdjxm\" (UID: \"6c277f91-c521-4062-bb13-d23bdeb073aa\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cdjxm" Jul 06 20:13:20.874131 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:20.874034 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24kwn\" (UniqueName: \"kubernetes.io/projected/6c277f91-c521-4062-bb13-d23bdeb073aa-kube-api-access-24kwn\") pod \"limitador-limitador-54d6d988d5-cdjxm\" (UID: \"6c277f91-c521-4062-bb13-d23bdeb073aa\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cdjxm" Jul 06 20:13:20.974688 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:20.974663 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6c277f91-c521-4062-bb13-d23bdeb073aa-config-file\") pod \"limitador-limitador-54d6d988d5-cdjxm\" (UID: \"6c277f91-c521-4062-bb13-d23bdeb073aa\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cdjxm" Jul 06 20:13:20.974775 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:20.974696 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24kwn\" (UniqueName: \"kubernetes.io/projected/6c277f91-c521-4062-bb13-d23bdeb073aa-kube-api-access-24kwn\") pod \"limitador-limitador-54d6d988d5-cdjxm\" (UID: \"6c277f91-c521-4062-bb13-d23bdeb073aa\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cdjxm" Jul 06 20:13:20.975291 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:20.975270 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6c277f91-c521-4062-bb13-d23bdeb073aa-config-file\") pod \"limitador-limitador-54d6d988d5-cdjxm\" (UID: \"6c277f91-c521-4062-bb13-d23bdeb073aa\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cdjxm" Jul 06 20:13:20.987002 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:20.986983 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24kwn\" (UniqueName: \"kubernetes.io/projected/6c277f91-c521-4062-bb13-d23bdeb073aa-kube-api-access-24kwn\") pod \"limitador-limitador-54d6d988d5-cdjxm\" (UID: \"6c277f91-c521-4062-bb13-d23bdeb073aa\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cdjxm" Jul 06 20:13:21.127440 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:21.127381 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-cdjxm" Jul 06 20:13:21.309578 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:21.309552 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:13:21.310477 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:13:21.310446 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c277f91_c521_4062_bb13_d23bdeb073aa.slice/crio-a023367373b4297ebe6cc5f461f280febbed2609f73f75da2d1440b0008ff97f WatchSource:0}: Error finding container a023367373b4297ebe6cc5f461f280febbed2609f73f75da2d1440b0008ff97f: Status 404 returned error can't find the container with id a023367373b4297ebe6cc5f461f280febbed2609f73f75da2d1440b0008ff97f Jul 06 20:13:21.994249 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:21.994210 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-cdjxm" event={"ID":"6c277f91-c521-4062-bb13-d23bdeb073aa","Type":"ContainerStarted","Data":"a023367373b4297ebe6cc5f461f280febbed2609f73f75da2d1440b0008ff97f"} Jul 06 20:13:25.016620 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:25.016578 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-cdjxm" event={"ID":"6c277f91-c521-4062-bb13-d23bdeb073aa","Type":"ContainerStarted","Data":"f8fa17b39ea2d60c4a3250783ff829ee1b606b0083232b1cc0060f8be9db1404"} Jul 06 20:13:25.017091 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:25.016738 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-cdjxm" Jul 06 20:13:25.039888 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:25.039824 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-cdjxm" podStartSLOduration=1.832724214 podStartE2EDuration="5.039810439s" totalImagesPullingTime="3.207086225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:13:25.036308481 +0000 UTC m=+458.163220585" watchObservedRunningTime="2026-07-06 20:13:25.039810439 +0000 UTC m=+458.166722543" Jul 06 20:13:36.026852 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:36.026823 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-cdjxm" Jul 06 20:13:55.257408 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:55.257377 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55896c4b85-mpzct"] Jul 06 20:13:55.265628 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:55.265600 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55896c4b85-mpzct" Jul 06 20:13:55.269246 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:55.269222 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dcbv8\"" Jul 06 20:13:55.269355 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:55.269229 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 20:13:55.275246 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:55.275227 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55896c4b85-mpzct"] Jul 06 20:13:55.359823 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:55.359793 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4d9f2e4e-c46f-40db-89d3-68ca87fac94b-tls-cert\") pod \"authorino-55896c4b85-mpzct\" (UID: \"4d9f2e4e-c46f-40db-89d3-68ca87fac94b\") " pod="kuadrant-system/authorino-55896c4b85-mpzct" Jul 06 20:13:55.359971 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:55.359955 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsnrg\" (UniqueName: \"kubernetes.io/projected/4d9f2e4e-c46f-40db-89d3-68ca87fac94b-kube-api-access-vsnrg\") pod \"authorino-55896c4b85-mpzct\" (UID: \"4d9f2e4e-c46f-40db-89d3-68ca87fac94b\") " pod="kuadrant-system/authorino-55896c4b85-mpzct" Jul 06 20:13:55.461113 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:55.461083 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsnrg\" (UniqueName: \"kubernetes.io/projected/4d9f2e4e-c46f-40db-89d3-68ca87fac94b-kube-api-access-vsnrg\") pod \"authorino-55896c4b85-mpzct\" (UID: \"4d9f2e4e-c46f-40db-89d3-68ca87fac94b\") " pod="kuadrant-system/authorino-55896c4b85-mpzct" Jul 06 20:13:55.461244 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:55.461132 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4d9f2e4e-c46f-40db-89d3-68ca87fac94b-tls-cert\") pod \"authorino-55896c4b85-mpzct\" (UID: \"4d9f2e4e-c46f-40db-89d3-68ca87fac94b\") " pod="kuadrant-system/authorino-55896c4b85-mpzct" Jul 06 20:13:55.463614 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:55.463589 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4d9f2e4e-c46f-40db-89d3-68ca87fac94b-tls-cert\") pod \"authorino-55896c4b85-mpzct\" (UID: \"4d9f2e4e-c46f-40db-89d3-68ca87fac94b\") " pod="kuadrant-system/authorino-55896c4b85-mpzct" Jul 06 20:13:55.473780 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:55.473760 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsnrg\" (UniqueName: \"kubernetes.io/projected/4d9f2e4e-c46f-40db-89d3-68ca87fac94b-kube-api-access-vsnrg\") pod \"authorino-55896c4b85-mpzct\" (UID: \"4d9f2e4e-c46f-40db-89d3-68ca87fac94b\") " pod="kuadrant-system/authorino-55896c4b85-mpzct" Jul 06 20:13:55.575789 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:55.575734 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55896c4b85-mpzct" Jul 06 20:13:55.712939 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:55.712906 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55896c4b85-mpzct"] Jul 06 20:13:55.714008 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:13:55.713983 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d9f2e4e_c46f_40db_89d3_68ca87fac94b.slice/crio-c36b6d7b4fe8283d0ec8e5030ca405f1c6c860b6537dbecb599e110a719c6309 WatchSource:0}: Error finding container c36b6d7b4fe8283d0ec8e5030ca405f1c6c860b6537dbecb599e110a719c6309: Status 404 returned error can't find the container with id c36b6d7b4fe8283d0ec8e5030ca405f1c6c860b6537dbecb599e110a719c6309 Jul 06 20:13:56.195570 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:13:56.195539 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55896c4b85-mpzct" event={"ID":"4d9f2e4e-c46f-40db-89d3-68ca87fac94b","Type":"ContainerStarted","Data":"c36b6d7b4fe8283d0ec8e5030ca405f1c6c860b6537dbecb599e110a719c6309"} Jul 06 20:14:00.220184 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:00.220151 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55896c4b85-mpzct" event={"ID":"4d9f2e4e-c46f-40db-89d3-68ca87fac94b","Type":"ContainerStarted","Data":"844c07dda1cdb6f9932f893ba99e30c9abefd75eeb9b64a865550536b0fa318f"} Jul 06 20:14:00.255294 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:00.255249 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-55896c4b85-mpzct" podStartSLOduration=1.452469462 podStartE2EDuration="5.255236018s" totalImagesPullingTime="3.802766556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:13:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:14:00.253090571 +0000 UTC m=+493.380002675" watchObservedRunningTime="2026-07-06 20:14:00.255236018 +0000 UTC m=+493.382148139" Jul 06 20:14:41.993983 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:41.993902 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:14:49.512417 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:49.512351 2583 generic.go:356] "Generic (PLEG): container finished" podID="1dd007b4-41d0-46a4-ba88-5ee40d497497" containerID="9f6a1b9db3e872a38c7c5815625831ed4a1ef22d98c4a329c8b718e61a2180a1" exitCode=0 Jul 06 20:14:49.512771 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:49.512429 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" event={"ID":"1dd007b4-41d0-46a4-ba88-5ee40d497497","Type":"ContainerDied","Data":"9f6a1b9db3e872a38c7c5815625831ed4a1ef22d98c4a329c8b718e61a2180a1"} Jul 06 20:14:49.512836 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:49.512819 2583 scope.go:117] "RemoveContainer" containerID="9f6a1b9db3e872a38c7c5815625831ed4a1ef22d98c4a329c8b718e61a2180a1" Jul 06 20:14:49.513809 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:49.513782 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:14:49.513889 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:49.513844 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:14:49.513933 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:49.513903 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:14:49.976425 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:49.976398 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:14:50.520052 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:50.520018 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" event={"ID":"1dd007b4-41d0-46a4-ba88-5ee40d497497","Type":"ContainerStarted","Data":"7e4785e279ee8db85d5e38af5ea3e0e374c39cc2516c58ed75a0c838f948bb04"} Jul 06 20:14:50.784965 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:50.784857 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:14:50.786129 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:50.786101 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jxm4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 20:14:50.786280 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:50.786161 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" podUID="1dd007b4-41d0-46a4-ba88-5ee40d497497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 20:14:51.785213 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:51.785181 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jxm4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 20:14:51.785617 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:51.785231 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" podUID="1dd007b4-41d0-46a4-ba88-5ee40d497497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 20:14:52.784776 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:52.784745 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jxm4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 20:14:52.784993 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:52.784799 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" podUID="1dd007b4-41d0-46a4-ba88-5ee40d497497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 20:14:53.785380 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:53.785347 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jxm4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 20:14:53.785762 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:53.785413 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" podUID="1dd007b4-41d0-46a4-ba88-5ee40d497497" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 20:14:54.788507 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:54.788479 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:14:54.788980 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:54.788922 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:14:54.789542 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:14:54.789524 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jxm4s" Jul 06 20:15:08.300948 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:15:08.300910 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:15:17.000962 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:15:17.000923 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:15:27.503451 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:15:27.503415 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:15:36.105448 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:15:36.105414 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:15:47.302607 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:15:47.302574 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:15:47.308494 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:15:47.308469 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:15:48.528418 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:15:48.528388 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:16:10.043521 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:16:10.043443 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:17:27.025523 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:27.025490 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d5958db9b-tfpxl"] Jul 06 20:17:27.029420 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:27.029397 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d5958db9b-tfpxl" Jul 06 20:17:27.041742 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:27.041722 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d5958db9b-tfpxl"] Jul 06 20:17:27.086549 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:27.086525 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b271f5aa-0263-47ea-9f03-b2dcbc80a78e-tls-cert\") pod \"authorino-5d5958db9b-tfpxl\" (UID: \"b271f5aa-0263-47ea-9f03-b2dcbc80a78e\") " pod="kuadrant-system/authorino-5d5958db9b-tfpxl" Jul 06 20:17:27.086678 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:27.086557 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf766\" (UniqueName: \"kubernetes.io/projected/b271f5aa-0263-47ea-9f03-b2dcbc80a78e-kube-api-access-wf766\") pod \"authorino-5d5958db9b-tfpxl\" (UID: \"b271f5aa-0263-47ea-9f03-b2dcbc80a78e\") " pod="kuadrant-system/authorino-5d5958db9b-tfpxl" Jul 06 20:17:27.188057 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:27.188032 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b271f5aa-0263-47ea-9f03-b2dcbc80a78e-tls-cert\") pod \"authorino-5d5958db9b-tfpxl\" (UID: \"b271f5aa-0263-47ea-9f03-b2dcbc80a78e\") " pod="kuadrant-system/authorino-5d5958db9b-tfpxl" Jul 06 20:17:27.188170 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:27.188061 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf766\" (UniqueName: \"kubernetes.io/projected/b271f5aa-0263-47ea-9f03-b2dcbc80a78e-kube-api-access-wf766\") pod \"authorino-5d5958db9b-tfpxl\" (UID: \"b271f5aa-0263-47ea-9f03-b2dcbc80a78e\") " pod="kuadrant-system/authorino-5d5958db9b-tfpxl" Jul 06 20:17:27.190406 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:27.190385 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b271f5aa-0263-47ea-9f03-b2dcbc80a78e-tls-cert\") pod \"authorino-5d5958db9b-tfpxl\" (UID: \"b271f5aa-0263-47ea-9f03-b2dcbc80a78e\") " pod="kuadrant-system/authorino-5d5958db9b-tfpxl" Jul 06 20:17:27.198751 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:27.198731 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf766\" (UniqueName: \"kubernetes.io/projected/b271f5aa-0263-47ea-9f03-b2dcbc80a78e-kube-api-access-wf766\") pod \"authorino-5d5958db9b-tfpxl\" (UID: \"b271f5aa-0263-47ea-9f03-b2dcbc80a78e\") " pod="kuadrant-system/authorino-5d5958db9b-tfpxl" Jul 06 20:17:27.339462 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:27.339429 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d5958db9b-tfpxl" Jul 06 20:17:27.478981 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:27.478959 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d5958db9b-tfpxl"] Jul 06 20:17:27.480780 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:17:27.480746 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb271f5aa_0263_47ea_9f03_b2dcbc80a78e.slice/crio-cc063bdb3d3957aba6858ae78e840a805c0a63306c9214bc218f95890d8fb12d WatchSource:0}: Error finding container cc063bdb3d3957aba6858ae78e840a805c0a63306c9214bc218f95890d8fb12d: Status 404 returned error can't find the container with id cc063bdb3d3957aba6858ae78e840a805c0a63306c9214bc218f95890d8fb12d Jul 06 20:17:27.482145 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:27.482119 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:17:28.434029 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:28.433931 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d5958db9b-tfpxl" event={"ID":"b271f5aa-0263-47ea-9f03-b2dcbc80a78e","Type":"ContainerStarted","Data":"625de90532c0b494013f68c5aeb7535caa89004f8fc8370d9982261a1f35de48"} Jul 06 20:17:28.434524 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:28.434212 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d5958db9b-tfpxl" event={"ID":"b271f5aa-0263-47ea-9f03-b2dcbc80a78e","Type":"ContainerStarted","Data":"cc063bdb3d3957aba6858ae78e840a805c0a63306c9214bc218f95890d8fb12d"} Jul 06 20:17:28.464559 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:28.464499 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d5958db9b-tfpxl" podStartSLOduration=0.815501345 podStartE2EDuration="1.464476757s" totalImagesPullingTime="648.975412ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:17:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:17:28.460105817 +0000 UTC m=+701.587017936" watchObservedRunningTime="2026-07-06 20:17:28.464476757 +0000 UTC m=+701.591388864" Jul 06 20:17:28.523895 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:28.520055 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55896c4b85-mpzct"] Jul 06 20:17:28.523895 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:28.521315 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-55896c4b85-mpzct" podUID="4d9f2e4e-c46f-40db-89d3-68ca87fac94b" containerName="authorino" containerID="cri-o://844c07dda1cdb6f9932f893ba99e30c9abefd75eeb9b64a865550536b0fa318f" gracePeriod=30 Jul 06 20:17:28.786146 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:28.786123 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55896c4b85-mpzct" Jul 06 20:17:28.904723 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:28.904691 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsnrg\" (UniqueName: \"kubernetes.io/projected/4d9f2e4e-c46f-40db-89d3-68ca87fac94b-kube-api-access-vsnrg\") pod \"4d9f2e4e-c46f-40db-89d3-68ca87fac94b\" (UID: \"4d9f2e4e-c46f-40db-89d3-68ca87fac94b\") " Jul 06 20:17:28.904921 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:28.904906 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4d9f2e4e-c46f-40db-89d3-68ca87fac94b-tls-cert\") pod \"4d9f2e4e-c46f-40db-89d3-68ca87fac94b\" (UID: \"4d9f2e4e-c46f-40db-89d3-68ca87fac94b\") " Jul 06 20:17:28.906785 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:28.906753 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d9f2e4e-c46f-40db-89d3-68ca87fac94b-kube-api-access-vsnrg" (OuterVolumeSpecName: "kube-api-access-vsnrg") pod "4d9f2e4e-c46f-40db-89d3-68ca87fac94b" (UID: "4d9f2e4e-c46f-40db-89d3-68ca87fac94b"). InnerVolumeSpecName "kube-api-access-vsnrg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:17:28.916259 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:28.916227 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d9f2e4e-c46f-40db-89d3-68ca87fac94b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "4d9f2e4e-c46f-40db-89d3-68ca87fac94b" (UID: "4d9f2e4e-c46f-40db-89d3-68ca87fac94b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:17:29.006067 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:29.006033 2583 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4d9f2e4e-c46f-40db-89d3-68ca87fac94b-tls-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:17:29.006067 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:29.006065 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsnrg\" (UniqueName: \"kubernetes.io/projected/4d9f2e4e-c46f-40db-89d3-68ca87fac94b-kube-api-access-vsnrg\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:17:29.442686 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:29.442594 2583 generic.go:356] "Generic (PLEG): container finished" podID="4d9f2e4e-c46f-40db-89d3-68ca87fac94b" containerID="844c07dda1cdb6f9932f893ba99e30c9abefd75eeb9b64a865550536b0fa318f" exitCode=0 Jul 06 20:17:29.442686 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:29.442639 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55896c4b85-mpzct" Jul 06 20:17:29.443249 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:29.442680 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55896c4b85-mpzct" event={"ID":"4d9f2e4e-c46f-40db-89d3-68ca87fac94b","Type":"ContainerDied","Data":"844c07dda1cdb6f9932f893ba99e30c9abefd75eeb9b64a865550536b0fa318f"} Jul 06 20:17:29.443249 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:29.442718 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55896c4b85-mpzct" event={"ID":"4d9f2e4e-c46f-40db-89d3-68ca87fac94b","Type":"ContainerDied","Data":"c36b6d7b4fe8283d0ec8e5030ca405f1c6c860b6537dbecb599e110a719c6309"} Jul 06 20:17:29.443249 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:29.442739 2583 scope.go:117] "RemoveContainer" containerID="844c07dda1cdb6f9932f893ba99e30c9abefd75eeb9b64a865550536b0fa318f" Jul 06 20:17:29.454324 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:29.454309 2583 scope.go:117] "RemoveContainer" containerID="844c07dda1cdb6f9932f893ba99e30c9abefd75eeb9b64a865550536b0fa318f" Jul 06 20:17:29.454554 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:17:29.454536 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"844c07dda1cdb6f9932f893ba99e30c9abefd75eeb9b64a865550536b0fa318f\": container with ID starting with 844c07dda1cdb6f9932f893ba99e30c9abefd75eeb9b64a865550536b0fa318f not found: ID does not exist" containerID="844c07dda1cdb6f9932f893ba99e30c9abefd75eeb9b64a865550536b0fa318f" Jul 06 20:17:29.454626 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:29.454560 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"844c07dda1cdb6f9932f893ba99e30c9abefd75eeb9b64a865550536b0fa318f"} err="failed to get container status \"844c07dda1cdb6f9932f893ba99e30c9abefd75eeb9b64a865550536b0fa318f\": rpc error: code = NotFound desc = could not find container \"844c07dda1cdb6f9932f893ba99e30c9abefd75eeb9b64a865550536b0fa318f\": container with ID starting with 844c07dda1cdb6f9932f893ba99e30c9abefd75eeb9b64a865550536b0fa318f not found: ID does not exist" Jul 06 20:17:29.471642 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:29.471614 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55896c4b85-mpzct"] Jul 06 20:17:29.487586 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:29.487565 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-55896c4b85-mpzct"] Jul 06 20:17:31.484800 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:31.484764 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d9f2e4e-c46f-40db-89d3-68ca87fac94b" path="/var/lib/kubelet/pods/4d9f2e4e-c46f-40db-89d3-68ca87fac94b/volumes" Jul 06 20:17:58.639267 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:17:58.639233 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:18:08.529636 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:18:08.529601 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:18:20.928493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:18:20.928457 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:18:29.160390 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:18:29.160356 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:18:40.622369 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:18:40.622334 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:19:21.559921 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:19:21.559890 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:19:33.749577 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:19:33.749540 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:19:54.027635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:19:54.027600 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:20:04.062852 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:20:04.062815 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:20:14.018234 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:20:14.018197 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:20:21.293795 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:20:21.293761 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:20:32.819898 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:20:32.819841 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:20:43.882332 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:20:43.882250 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:20:47.344787 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:20:47.344759 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:20:47.350376 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:20:47.350350 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:20:54.990207 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:20:54.990170 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:21:04.800332 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:21:04.800295 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:21:27.417407 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:21:27.417367 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:21:37.194958 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:21:37.194928 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:22:02.896450 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:22:02.896417 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:22:13.011589 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:22:13.011513 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:22:28.897819 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:22:28.897785 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:22:39.534468 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:22:39.534426 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:22:48.804418 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:22:48.804386 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:22:58.303744 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:22:58.303708 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:23:06.717946 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:06.717907 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:23:34.144619 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:34.144581 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml"] Jul 06 20:23:34.145196 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:34.144796 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" podUID="6d5676ab-0efc-40b0-9250-e73143cb417d" containerName="manager" containerID="cri-o://787f895c8a7173b6032548f57bd4d967205b6c9034ed3fb44f2fc01440eb93e1" gracePeriod=10 Jul 06 20:23:36.588536 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:36.588507 2583 generic.go:356] "Generic (PLEG): container finished" podID="6d5676ab-0efc-40b0-9250-e73143cb417d" containerID="787f895c8a7173b6032548f57bd4d967205b6c9034ed3fb44f2fc01440eb93e1" exitCode=0 Jul 06 20:23:36.588847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:36.588586 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" event={"ID":"6d5676ab-0efc-40b0-9250-e73143cb417d","Type":"ContainerDied","Data":"787f895c8a7173b6032548f57bd4d967205b6c9034ed3fb44f2fc01440eb93e1"} Jul 06 20:23:36.713041 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:36.713015 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" Jul 06 20:23:36.882030 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:36.881949 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d5676ab-0efc-40b0-9250-e73143cb417d-extensions-socket-volume\") pod \"6d5676ab-0efc-40b0-9250-e73143cb417d\" (UID: \"6d5676ab-0efc-40b0-9250-e73143cb417d\") " Jul 06 20:23:36.882030 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:36.881989 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcjlh\" (UniqueName: \"kubernetes.io/projected/6d5676ab-0efc-40b0-9250-e73143cb417d-kube-api-access-gcjlh\") pod \"6d5676ab-0efc-40b0-9250-e73143cb417d\" (UID: \"6d5676ab-0efc-40b0-9250-e73143cb417d\") " Jul 06 20:23:36.882405 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:36.882383 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d5676ab-0efc-40b0-9250-e73143cb417d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6d5676ab-0efc-40b0-9250-e73143cb417d" (UID: "6d5676ab-0efc-40b0-9250-e73143cb417d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:23:36.884069 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:36.884033 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d5676ab-0efc-40b0-9250-e73143cb417d-kube-api-access-gcjlh" (OuterVolumeSpecName: "kube-api-access-gcjlh") pod "6d5676ab-0efc-40b0-9250-e73143cb417d" (UID: "6d5676ab-0efc-40b0-9250-e73143cb417d"). InnerVolumeSpecName "kube-api-access-gcjlh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:23:36.982810 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:36.982784 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d5676ab-0efc-40b0-9250-e73143cb417d-extensions-socket-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:23:36.982810 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:36.982806 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcjlh\" (UniqueName: \"kubernetes.io/projected/6d5676ab-0efc-40b0-9250-e73143cb417d-kube-api-access-gcjlh\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:23:37.595134 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:37.595093 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" event={"ID":"6d5676ab-0efc-40b0-9250-e73143cb417d","Type":"ContainerDied","Data":"15e08a0e821bcdfdcc3832a93958f96928701399de95ea5124c6622abd1b7e46"} Jul 06 20:23:37.595134 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:37.595122 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml" Jul 06 20:23:37.595606 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:37.595143 2583 scope.go:117] "RemoveContainer" containerID="787f895c8a7173b6032548f57bd4d967205b6c9034ed3fb44f2fc01440eb93e1" Jul 06 20:23:37.626167 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:37.626144 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml"] Jul 06 20:23:37.631676 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:37.631657 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwrml"] Jul 06 20:23:39.485243 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:23:39.485171 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d5676ab-0efc-40b0-9250-e73143cb417d" path="/var/lib/kubelet/pods/6d5676ab-0efc-40b0-9250-e73143cb417d/volumes" Jul 06 20:24:35.395226 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.395179 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9"] Jul 06 20:24:35.395696 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.395608 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d5676ab-0efc-40b0-9250-e73143cb417d" containerName="manager" Jul 06 20:24:35.395696 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.395620 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d5676ab-0efc-40b0-9250-e73143cb417d" containerName="manager" Jul 06 20:24:35.395696 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.395642 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d9f2e4e-c46f-40db-89d3-68ca87fac94b" containerName="authorino" Jul 06 20:24:35.395696 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.395648 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d9f2e4e-c46f-40db-89d3-68ca87fac94b" containerName="authorino" Jul 06 20:24:35.395837 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.395703 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d5676ab-0efc-40b0-9250-e73143cb417d" containerName="manager" Jul 06 20:24:35.395837 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.395711 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d9f2e4e-c46f-40db-89d3-68ca87fac94b" containerName="authorino" Jul 06 20:24:35.398923 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.398903 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" Jul 06 20:24:35.401272 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.401251 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-pn8tl\"" Jul 06 20:24:35.411324 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.411302 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9"] Jul 06 20:24:35.539781 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.539758 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbg22\" (UniqueName: \"kubernetes.io/projected/a9399ca9-ef00-49bc-97f9-ce45134fed9d-kube-api-access-fbg22\") pod \"kuadrant-operator-controller-manager-86878c769f-pzfd9\" (UID: \"a9399ca9-ef00-49bc-97f9-ce45134fed9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" Jul 06 20:24:35.539927 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.539796 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9399ca9-ef00-49bc-97f9-ce45134fed9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pzfd9\" (UID: \"a9399ca9-ef00-49bc-97f9-ce45134fed9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" Jul 06 20:24:35.640616 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.640587 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9399ca9-ef00-49bc-97f9-ce45134fed9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pzfd9\" (UID: \"a9399ca9-ef00-49bc-97f9-ce45134fed9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" Jul 06 20:24:35.640739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.640701 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbg22\" (UniqueName: \"kubernetes.io/projected/a9399ca9-ef00-49bc-97f9-ce45134fed9d-kube-api-access-fbg22\") pod \"kuadrant-operator-controller-manager-86878c769f-pzfd9\" (UID: \"a9399ca9-ef00-49bc-97f9-ce45134fed9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" Jul 06 20:24:35.641020 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.640998 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9399ca9-ef00-49bc-97f9-ce45134fed9d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pzfd9\" (UID: \"a9399ca9-ef00-49bc-97f9-ce45134fed9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" Jul 06 20:24:35.649758 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.649705 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbg22\" (UniqueName: \"kubernetes.io/projected/a9399ca9-ef00-49bc-97f9-ce45134fed9d-kube-api-access-fbg22\") pod \"kuadrant-operator-controller-manager-86878c769f-pzfd9\" (UID: \"a9399ca9-ef00-49bc-97f9-ce45134fed9d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" Jul 06 20:24:35.709452 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.709429 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" Jul 06 20:24:35.851562 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.851447 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9"] Jul 06 20:24:35.854184 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:24:35.854155 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9399ca9_ef00_49bc_97f9_ce45134fed9d.slice/crio-1a0bc1b88437ac69d7106813fec6e998e4461ac2098f6a61d6626b7778c3d499 WatchSource:0}: Error finding container 1a0bc1b88437ac69d7106813fec6e998e4461ac2098f6a61d6626b7778c3d499: Status 404 returned error can't find the container with id 1a0bc1b88437ac69d7106813fec6e998e4461ac2098f6a61d6626b7778c3d499 Jul 06 20:24:35.927228 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.927198 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" event={"ID":"a9399ca9-ef00-49bc-97f9-ce45134fed9d","Type":"ContainerStarted","Data":"6a3743110830313d7e5d69e046db8c234c81c8659ffbcda632ed020bd4e8c5f9"} Jul 06 20:24:35.927377 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.927360 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" Jul 06 20:24:35.927442 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.927401 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" event={"ID":"a9399ca9-ef00-49bc-97f9-ce45134fed9d","Type":"ContainerStarted","Data":"1a0bc1b88437ac69d7106813fec6e998e4461ac2098f6a61d6626b7778c3d499"} Jul 06 20:24:35.947399 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:35.947356 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" podStartSLOduration=0.947341183 podStartE2EDuration="947.341183ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:24:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:35.947070644 +0000 UTC m=+1129.073982752" watchObservedRunningTime="2026-07-06 20:24:35.947341183 +0000 UTC m=+1129.074253287" Jul 06 20:24:46.934481 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:46.934397 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" Jul 06 20:24:56.792932 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:24:56.792901 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:25:07.712272 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:25:07.712242 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:25:18.612957 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:25:18.612921 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:25:27.515220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:25:27.515185 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:25:38.306682 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:25:38.306648 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:25:45.870287 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:25:45.870251 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:25:47.391446 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:25:47.391412 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:25:47.392796 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:25:47.392774 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:25:56.210719 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:25:56.210681 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:26:06.394073 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:26:06.394036 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:26:16.726041 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:26:16.726001 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:26:26.912685 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:26:26.912646 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:26:38.197094 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:26:38.197059 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:26:45.203079 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:26:45.203001 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:27:04.229226 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:27:04.229191 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:27:13.119959 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:27:13.119925 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:27:24.216651 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:27:24.216616 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:27:33.512303 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:27:33.512271 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:27:50.033419 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:27:50.033376 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:27:59.110361 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:27:59.110324 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:28:11.417292 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:28:11.417211 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:28:21.913931 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:28:21.913896 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:28:30.710537 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:28:30.710501 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:28:41.713588 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:28:41.713557 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:28:54.540510 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:28:54.540480 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:29:03.606467 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:29:03.606433 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:29:12.834528 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:29:12.834476 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:29:24.012495 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:29:24.012457 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:29:36.602282 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:29:36.602248 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:29:44.341904 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:29:44.341806 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:29:53.526852 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:29:53.526819 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:30:00.169886 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:00.169843 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp"] Jul 06 20:30:00.173727 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:00.173711 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp" Jul 06 20:30:00.179161 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:00.179136 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hsq27\"" Jul 06 20:30:00.179354 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:00.179331 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 06 20:30:00.180020 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:00.179997 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 06 20:30:00.198475 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:00.198451 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp"] Jul 06 20:30:00.297806 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:00.297768 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr5rz\" (UniqueName: \"kubernetes.io/projected/e8e28657-8eea-41ff-adce-53cf00ba48d2-kube-api-access-qr5rz\") pod \"maas-api-key-cleanup-29722830-4w4wp\" (UID: \"e8e28657-8eea-41ff-adce-53cf00ba48d2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp" Jul 06 20:30:00.399070 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:00.399037 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr5rz\" (UniqueName: \"kubernetes.io/projected/e8e28657-8eea-41ff-adce-53cf00ba48d2-kube-api-access-qr5rz\") pod \"maas-api-key-cleanup-29722830-4w4wp\" (UID: \"e8e28657-8eea-41ff-adce-53cf00ba48d2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp" Jul 06 20:30:00.410267 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:00.410241 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr5rz\" (UniqueName: \"kubernetes.io/projected/e8e28657-8eea-41ff-adce-53cf00ba48d2-kube-api-access-qr5rz\") pod \"maas-api-key-cleanup-29722830-4w4wp\" (UID: \"e8e28657-8eea-41ff-adce-53cf00ba48d2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp" Jul 06 20:30:00.491847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:00.491806 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp" Jul 06 20:30:00.838407 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:00.838380 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp"] Jul 06 20:30:00.839302 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:30:00.839276 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8e28657_8eea_41ff_adce_53cf00ba48d2.slice/crio-a406e54af5eea6218932476c6b43eb650f8826f443b15f3f3dad530b30bc695e WatchSource:0}: Error finding container a406e54af5eea6218932476c6b43eb650f8826f443b15f3f3dad530b30bc695e: Status 404 returned error can't find the container with id a406e54af5eea6218932476c6b43eb650f8826f443b15f3f3dad530b30bc695e Jul 06 20:30:00.841519 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:00.841496 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:30:01.809761 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:01.809728 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp" event={"ID":"e8e28657-8eea-41ff-adce-53cf00ba48d2","Type":"ContainerStarted","Data":"a406e54af5eea6218932476c6b43eb650f8826f443b15f3f3dad530b30bc695e"} Jul 06 20:30:04.831213 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:04.831173 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp" event={"ID":"e8e28657-8eea-41ff-adce-53cf00ba48d2","Type":"ContainerStarted","Data":"58aff442d7e5b37f6503196620ff15b92fa23174f87f546eb168bed8b4d23bf5"} Jul 06 20:30:04.854845 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:04.854792 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp" podStartSLOduration=1.768862924 podStartE2EDuration="4.854777042s" totalImagesPullingTime="3.085914118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:30:04.852635242 +0000 UTC m=+1457.979547358" watchObservedRunningTime="2026-07-06 20:30:04.854777042 +0000 UTC m=+1457.981689146" Jul 06 20:30:10.342986 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:10.342949 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:30:19.651279 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:19.651247 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:30:33.662745 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:33.662706 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:30:44.956635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:44.956567 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" podUID="a5f8a799-9d7a-498c-8e6d-8baa3036efab" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.30:8081/readyz\": dial tcp 10.132.0.30:8081: connect: connection refused" Jul 06 20:30:45.067351 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.067322 2583 generic.go:356] "Generic (PLEG): container finished" podID="ed7b2a0f-049e-4874-8a09-076638035822" containerID="23d0afd561fcd9931c3b983083f8bb179d1169e347d4de2cf68c5c4e93856a4b" exitCode=1 Jul 06 20:30:45.067485 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.067396 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-np686" event={"ID":"ed7b2a0f-049e-4874-8a09-076638035822","Type":"ContainerDied","Data":"23d0afd561fcd9931c3b983083f8bb179d1169e347d4de2cf68c5c4e93856a4b"} Jul 06 20:30:45.067798 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.067778 2583 scope.go:117] "RemoveContainer" containerID="23d0afd561fcd9931c3b983083f8bb179d1169e347d4de2cf68c5c4e93856a4b" Jul 06 20:30:45.069132 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.069112 2583 generic.go:356] "Generic (PLEG): container finished" podID="0732e11e-ca32-46dc-b9fe-a7ad6ef72eae" containerID="029fe1b84440bf71443f633f66ff7d3a5c303ef91d64bebc378485007a1b264f" exitCode=1 Jul 06 20:30:45.069233 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.069175 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk" event={"ID":"0732e11e-ca32-46dc-b9fe-a7ad6ef72eae","Type":"ContainerDied","Data":"029fe1b84440bf71443f633f66ff7d3a5c303ef91d64bebc378485007a1b264f"} Jul 06 20:30:45.069469 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.069455 2583 scope.go:117] "RemoveContainer" containerID="029fe1b84440bf71443f633f66ff7d3a5c303ef91d64bebc378485007a1b264f" Jul 06 20:30:45.070734 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.070714 2583 generic.go:356] "Generic (PLEG): container finished" podID="fb33694d-e0bd-479a-89cc-1cdc2be67151" containerID="029a6d438ef50972f58149d9b04806e12358ec70b17d8336e3fd840ba4926967" exitCode=1 Jul 06 20:30:45.070823 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.070785 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg" event={"ID":"fb33694d-e0bd-479a-89cc-1cdc2be67151","Type":"ContainerDied","Data":"029a6d438ef50972f58149d9b04806e12358ec70b17d8336e3fd840ba4926967"} Jul 06 20:30:45.071056 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.071037 2583 scope.go:117] "RemoveContainer" containerID="029a6d438ef50972f58149d9b04806e12358ec70b17d8336e3fd840ba4926967" Jul 06 20:30:45.072316 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.072297 2583 generic.go:356] "Generic (PLEG): container finished" podID="73340a04-d613-40d5-85ab-7c54daeaa757" containerID="afb463ca0f1bf8818ca70fa27ceb2a33e2d842f6043b88311d8474c7b53ffa4b" exitCode=1 Jul 06 20:30:45.072409 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.072352 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" event={"ID":"73340a04-d613-40d5-85ab-7c54daeaa757","Type":"ContainerDied","Data":"afb463ca0f1bf8818ca70fa27ceb2a33e2d842f6043b88311d8474c7b53ffa4b"} Jul 06 20:30:45.072668 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.072653 2583 scope.go:117] "RemoveContainer" containerID="afb463ca0f1bf8818ca70fa27ceb2a33e2d842f6043b88311d8474c7b53ffa4b" Jul 06 20:30:45.073024 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.073006 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 20:30:45.074538 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.074513 2583 generic.go:356] "Generic (PLEG): container finished" podID="a5f8a799-9d7a-498c-8e6d-8baa3036efab" containerID="be1c990f5f454f355177d1a746a3205e3204490d2d9570bb4afb6a94672bc5d4" exitCode=1 Jul 06 20:30:45.074653 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.074575 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" event={"ID":"a5f8a799-9d7a-498c-8e6d-8baa3036efab","Type":"ContainerDied","Data":"be1c990f5f454f355177d1a746a3205e3204490d2d9570bb4afb6a94672bc5d4"} Jul 06 20:30:45.074863 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:45.074845 2583 scope.go:117] "RemoveContainer" containerID="be1c990f5f454f355177d1a746a3205e3204490d2d9570bb4afb6a94672bc5d4" Jul 06 20:30:45.627397 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:30:45.627351 2583 controller.go:195] "Failed to update lease" err="Put \"https://ab9f98892658242508f171bd05a809f2-b2773de990c4921d.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-130-133.ec2.internal?timeout=10s\": context deadline exceeded" Jul 06 20:30:46.076886 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.076786 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-np686" Jul 06 20:30:46.082382 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.082355 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" event={"ID":"73340a04-d613-40d5-85ab-7c54daeaa757","Type":"ContainerStarted","Data":"42623994b4ff0b7b261ea4fb869a93799c8b8d509c0d1279451d275863dd6380"} Jul 06 20:30:46.082559 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.082544 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" Jul 06 20:30:46.084696 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.084673 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" event={"ID":"a5f8a799-9d7a-498c-8e6d-8baa3036efab","Type":"ContainerStarted","Data":"a862cd53d379fbb0bc01d9f027d3034d4f609b678d0f5aa7e416b0ecca3769ed"} Jul 06 20:30:46.084948 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.084932 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:30:46.086207 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.086186 2583 generic.go:356] "Generic (PLEG): container finished" podID="4c49034e-79c9-407b-b068-96bbe8d61dac" containerID="2cefd9c066ddafadc66a5c40282b48dfe5c24c02b24ba6b81d7813735167847b" exitCode=1 Jul 06 20:30:46.086289 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.086258 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" event={"ID":"4c49034e-79c9-407b-b068-96bbe8d61dac","Type":"ContainerDied","Data":"2cefd9c066ddafadc66a5c40282b48dfe5c24c02b24ba6b81d7813735167847b"} Jul 06 20:30:46.086532 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.086515 2583 scope.go:117] "RemoveContainer" containerID="2cefd9c066ddafadc66a5c40282b48dfe5c24c02b24ba6b81d7813735167847b" Jul 06 20:30:46.087861 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.087841 2583 generic.go:356] "Generic (PLEG): container finished" podID="ba75fdd1-3a5a-46cb-a655-a46013cf495e" containerID="c91c4ebe9375454a74bcba256ead184e4d2b92ca8572215e5e026fb01df87266" exitCode=1 Jul 06 20:30:46.087962 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.087916 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" event={"ID":"ba75fdd1-3a5a-46cb-a655-a46013cf495e","Type":"ContainerDied","Data":"c91c4ebe9375454a74bcba256ead184e4d2b92ca8572215e5e026fb01df87266"} Jul 06 20:30:46.088162 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.088149 2583 scope.go:117] "RemoveContainer" containerID="c91c4ebe9375454a74bcba256ead184e4d2b92ca8572215e5e026fb01df87266" Jul 06 20:30:46.089921 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.089901 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-np686" event={"ID":"ed7b2a0f-049e-4874-8a09-076638035822","Type":"ContainerStarted","Data":"31a762e8d0783b6094a42d969e729bbbac59101c6bde52af3918483ada7af93c"} Jul 06 20:30:46.090430 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.090414 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-np686" Jul 06 20:30:46.092038 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.092015 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk" event={"ID":"0732e11e-ca32-46dc-b9fe-a7ad6ef72eae","Type":"ContainerStarted","Data":"2258278968e73fe8db1703f17ff3d4151b0754a17c0fa2996d8e7b3374db9717"} Jul 06 20:30:46.092213 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.092195 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk" Jul 06 20:30:46.097530 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.097512 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg" event={"ID":"fb33694d-e0bd-479a-89cc-1cdc2be67151","Type":"ContainerStarted","Data":"c06d3b63553b21f44009e933c3bf5d6129cee664fe3eed7ae04646cc1a321aae"} Jul 06 20:30:46.097707 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.097691 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg" Jul 06 20:30:46.099320 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.099303 2583 generic.go:356] "Generic (PLEG): container finished" podID="a9399ca9-ef00-49bc-97f9-ce45134fed9d" containerID="6a3743110830313d7e5d69e046db8c234c81c8659ffbcda632ed020bd4e8c5f9" exitCode=0 Jul 06 20:30:46.099416 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.099346 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" event={"ID":"a9399ca9-ef00-49bc-97f9-ce45134fed9d","Type":"ContainerDied","Data":"6a3743110830313d7e5d69e046db8c234c81c8659ffbcda632ed020bd4e8c5f9"} Jul 06 20:30:46.099639 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.099624 2583 scope.go:117] "RemoveContainer" containerID="6a3743110830313d7e5d69e046db8c234c81c8659ffbcda632ed020bd4e8c5f9" Jul 06 20:30:46.144441 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.144420 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" Jul 06 20:30:46.677851 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.677824 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" Jul 06 20:30:46.933082 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:46.933052 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" Jul 06 20:30:47.107544 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:47.107506 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" event={"ID":"a9399ca9-ef00-49bc-97f9-ce45134fed9d","Type":"ContainerStarted","Data":"a7644ada750d3254f0a09dab1fde643060fc75f5ca1327e4367cb083f328acd6"} Jul 06 20:30:47.107970 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:47.107569 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" Jul 06 20:30:47.109361 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:47.109330 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" event={"ID":"4c49034e-79c9-407b-b068-96bbe8d61dac","Type":"ContainerStarted","Data":"c751fb6b929b0ff7bb4f67559b32008c668a088b77247561567f4bb65b896d05"} Jul 06 20:30:47.109496 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:47.109388 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" Jul 06 20:30:47.111175 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:47.111147 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" event={"ID":"ba75fdd1-3a5a-46cb-a655-a46013cf495e","Type":"ContainerStarted","Data":"6dd081f63f096242d4a3be28f4d4d11867d44add8a7d2ef9f07a8e8e225eff3c"} Jul 06 20:30:47.111518 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:47.111501 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" Jul 06 20:30:47.434263 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:47.434239 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:30:47.437080 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:47.437057 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:30:47.519755 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:30:47.519724 2583 controller.go:195] "Failed to update lease" err="Operation cannot be fulfilled on leases.coordination.k8s.io \"ip-10-0-130-133.ec2.internal\": the object has been modified; please apply your changes to the latest version and try again" Jul 06 20:30:57.113007 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:57.112977 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2k7bg" Jul 06 20:30:57.113426 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:57.113047 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-drtjk" Jul 06 20:30:57.113426 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:57.113078 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-np686" Jul 06 20:30:57.113637 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:57.113620 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-68cbf59454-jvfwm" Jul 06 20:30:58.119424 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:58.119396 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pzfd9" Jul 06 20:30:59.125711 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:30:59.125666 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tj6k6" Jul 06 20:31:00.029722 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:00.029689 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp"] Jul 06 20:31:00.029929 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:00.029907 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp" podUID="e8e28657-8eea-41ff-adce-53cf00ba48d2" containerName="cleanup" containerID="cri-o://58aff442d7e5b37f6503196620ff15b92fa23174f87f546eb168bed8b4d23bf5" gracePeriod=30 Jul 06 20:31:17.116527 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:17.116498 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bfvds" Jul 06 20:31:18.119299 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:18.119268 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vpbr" Jul 06 20:31:19.631663 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.631627 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m"] Jul 06 20:31:19.636280 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.636258 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.639596 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.639574 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8a8b3310-openshift-default-dockercfg-wmwz6\"" Jul 06 20:31:19.652840 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.652818 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m"] Jul 06 20:31:19.676888 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.676845 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-credential-socket\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.676888 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.676936 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-workload-certs\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.677368 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.676972 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-workload-socket\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.677368 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.677086 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-data\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.677368 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.677146 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istiod-ca-cert\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.677545 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.677488 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cc77\" (UniqueName: \"kubernetes.io/projected/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-kube-api-access-8cc77\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.677545 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.677528 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-envoy\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.677655 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.677563 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-podinfo\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.677709 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.677651 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-token\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.726367 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.726343 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:31:19.778224 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.778198 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-data\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.778368 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.778235 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istiod-ca-cert\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.778368 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.778260 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cc77\" (UniqueName: \"kubernetes.io/projected/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-kube-api-access-8cc77\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.778368 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.778278 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-envoy\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.778368 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.778298 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-podinfo\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.778596 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.778418 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-token\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.778596 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.778502 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-credential-socket\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.778596 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.778528 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-workload-certs\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.778596 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.778555 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-workload-socket\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.778814 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.778644 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-data\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.778814 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.778800 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-credential-socket\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.778984 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.778922 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-workload-socket\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.778984 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.778969 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-workload-certs\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.779089 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.779069 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istiod-ca-cert\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.780632 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.780610 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-envoy\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.780952 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.780936 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-podinfo\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.788407 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.788379 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-token\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.788600 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.788580 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cc77\" (UniqueName: \"kubernetes.io/projected/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-kube-api-access-8cc77\") pod \"e2e-ait-8a8b3310-openshift-default-895849675-ngs7m\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:19.949283 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:19.949225 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:20.101037 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:20.101010 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m"] Jul 06 20:31:20.101924 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:31:20.101901 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77c8ee68_97b1_4a7f_8c94_fefcc0b8d349.slice/crio-bc914240e877160d77eddee0cf38846996ebacf529720970897f5b99730586d4 WatchSource:0}: Error finding container bc914240e877160d77eddee0cf38846996ebacf529720970897f5b99730586d4: Status 404 returned error can't find the container with id bc914240e877160d77eddee0cf38846996ebacf529720970897f5b99730586d4 Jul 06 20:31:20.104199 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:20.104163 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:31:20.104278 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:20.104227 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:31:20.104278 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:20.104255 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:31:20.314072 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:20.314037 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" event={"ID":"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349","Type":"ContainerStarted","Data":"2f28a582263e623d607c8c7eaeba18fbe6a67eb0592a5f39a610a2e716840420"} Jul 06 20:31:20.314250 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:20.314099 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" event={"ID":"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349","Type":"ContainerStarted","Data":"bc914240e877160d77eddee0cf38846996ebacf529720970897f5b99730586d4"} Jul 06 20:31:20.343766 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:20.343715 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" podStartSLOduration=1.3436995440000001 podStartE2EDuration="1.343699544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:31:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:31:20.341650025 +0000 UTC m=+1533.468562133" watchObservedRunningTime="2026-07-06 20:31:20.343699544 +0000 UTC m=+1533.470611648" Jul 06 20:31:20.949777 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:20.949752 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:21.964859 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:21.964829 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:22.326280 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:22.326253 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:22.327086 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:22.327056 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:26.117297 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:26.117261 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m"] Jul 06 20:31:26.117695 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:26.117456 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" podUID="77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" containerName="istio-proxy" containerID="cri-o://2f28a582263e623d607c8c7eaeba18fbe6a67eb0592a5f39a610a2e716840420" gracePeriod=30 Jul 06 20:31:30.216908 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:30.216867 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp" Jul 06 20:31:30.280908 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:30.280861 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qr5rz\" (UniqueName: \"kubernetes.io/projected/e8e28657-8eea-41ff-adce-53cf00ba48d2-kube-api-access-qr5rz\") pod \"e8e28657-8eea-41ff-adce-53cf00ba48d2\" (UID: \"e8e28657-8eea-41ff-adce-53cf00ba48d2\") " Jul 06 20:31:30.283217 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:30.283194 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8e28657-8eea-41ff-adce-53cf00ba48d2-kube-api-access-qr5rz" (OuterVolumeSpecName: "kube-api-access-qr5rz") pod "e8e28657-8eea-41ff-adce-53cf00ba48d2" (UID: "e8e28657-8eea-41ff-adce-53cf00ba48d2"). InnerVolumeSpecName "kube-api-access-qr5rz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:31:30.380637 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:30.380602 2583 generic.go:356] "Generic (PLEG): container finished" podID="e8e28657-8eea-41ff-adce-53cf00ba48d2" containerID="58aff442d7e5b37f6503196620ff15b92fa23174f87f546eb168bed8b4d23bf5" exitCode=137 Jul 06 20:31:30.380805 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:30.380659 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp" Jul 06 20:31:30.380805 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:30.380694 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp" event={"ID":"e8e28657-8eea-41ff-adce-53cf00ba48d2","Type":"ContainerDied","Data":"58aff442d7e5b37f6503196620ff15b92fa23174f87f546eb168bed8b4d23bf5"} Jul 06 20:31:30.380805 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:30.380800 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp" event={"ID":"e8e28657-8eea-41ff-adce-53cf00ba48d2","Type":"ContainerDied","Data":"a406e54af5eea6218932476c6b43eb650f8826f443b15f3f3dad530b30bc695e"} Jul 06 20:31:30.380957 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:30.380830 2583 scope.go:117] "RemoveContainer" containerID="58aff442d7e5b37f6503196620ff15b92fa23174f87f546eb168bed8b4d23bf5" Jul 06 20:31:30.381734 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:30.381714 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qr5rz\" (UniqueName: \"kubernetes.io/projected/e8e28657-8eea-41ff-adce-53cf00ba48d2-kube-api-access-qr5rz\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:30.390912 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:30.390892 2583 scope.go:117] "RemoveContainer" containerID="58aff442d7e5b37f6503196620ff15b92fa23174f87f546eb168bed8b4d23bf5" Jul 06 20:31:30.391195 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:31:30.391169 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58aff442d7e5b37f6503196620ff15b92fa23174f87f546eb168bed8b4d23bf5\": container with ID starting with 58aff442d7e5b37f6503196620ff15b92fa23174f87f546eb168bed8b4d23bf5 not found: ID does not exist" containerID="58aff442d7e5b37f6503196620ff15b92fa23174f87f546eb168bed8b4d23bf5" Jul 06 20:31:30.391273 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:30.391205 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58aff442d7e5b37f6503196620ff15b92fa23174f87f546eb168bed8b4d23bf5"} err="failed to get container status \"58aff442d7e5b37f6503196620ff15b92fa23174f87f546eb168bed8b4d23bf5\": rpc error: code = NotFound desc = could not find container \"58aff442d7e5b37f6503196620ff15b92fa23174f87f546eb168bed8b4d23bf5\": container with ID starting with 58aff442d7e5b37f6503196620ff15b92fa23174f87f546eb168bed8b4d23bf5 not found: ID does not exist" Jul 06 20:31:30.406775 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:30.406614 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp"] Jul 06 20:31:30.409327 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:30.409306 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722830-4w4wp"] Jul 06 20:31:31.486423 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:31.486394 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8e28657-8eea-41ff-adce-53cf00ba48d2" path="/var/lib/kubelet/pods/e8e28657-8eea-41ff-adce-53cf00ba48d2/volumes" Jul 06 20:31:32.680493 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.680472 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:32.804566 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.804535 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-token\") pod \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " Jul 06 20:31:32.804739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.804579 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cc77\" (UniqueName: \"kubernetes.io/projected/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-kube-api-access-8cc77\") pod \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " Jul 06 20:31:32.804739 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.804700 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istiod-ca-cert\") pod \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " Jul 06 20:31:32.804869 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.804752 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-workload-certs\") pod \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " Jul 06 20:31:32.804869 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.804793 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-podinfo\") pod \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " Jul 06 20:31:32.804869 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.804829 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-credential-socket\") pod \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " Jul 06 20:31:32.805070 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.804910 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-workload-socket\") pod \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " Jul 06 20:31:32.805070 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.804940 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-envoy\") pod \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " Jul 06 20:31:32.805070 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.804971 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-data\") pod \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\" (UID: \"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349\") " Jul 06 20:31:32.805070 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.805032 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" (UID: "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:31:32.805260 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.805130 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" (UID: "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:31:32.805260 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.805159 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" (UID: "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:31:32.805260 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.805124 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" (UID: "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:31:32.805427 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.805380 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:32.805427 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.805399 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:32.805427 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.805414 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:32.805582 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.805428 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:32.806150 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.806123 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-data" (OuterVolumeSpecName: "istio-data") pod "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" (UID: "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:31:32.807036 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.807010 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" (UID: "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:31:32.807036 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.807018 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-kube-api-access-8cc77" (OuterVolumeSpecName: "kube-api-access-8cc77") pod "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" (UID: "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349"). InnerVolumeSpecName "kube-api-access-8cc77". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:31:32.807036 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.807029 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-token" (OuterVolumeSpecName: "istio-token") pod "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" (UID: "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:31:32.807192 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.807047 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" (UID: "77c8ee68-97b1-4a7f-8c94-fefcc0b8d349"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:31:32.906377 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.906355 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:32.906473 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.906379 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cc77\" (UniqueName: \"kubernetes.io/projected/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-kube-api-access-8cc77\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:32.906473 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.906395 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:32.906473 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.906407 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:32.906473 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:32.906419 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:33.400107 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.400074 2583 generic.go:356] "Generic (PLEG): container finished" podID="77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" containerID="2f28a582263e623d607c8c7eaeba18fbe6a67eb0592a5f39a610a2e716840420" exitCode=0 Jul 06 20:31:33.400266 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.400125 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" Jul 06 20:31:33.400266 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.400159 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" event={"ID":"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349","Type":"ContainerDied","Data":"2f28a582263e623d607c8c7eaeba18fbe6a67eb0592a5f39a610a2e716840420"} Jul 06 20:31:33.400266 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.400210 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m" event={"ID":"77c8ee68-97b1-4a7f-8c94-fefcc0b8d349","Type":"ContainerDied","Data":"bc914240e877160d77eddee0cf38846996ebacf529720970897f5b99730586d4"} Jul 06 20:31:33.400266 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.400231 2583 scope.go:117] "RemoveContainer" containerID="2f28a582263e623d607c8c7eaeba18fbe6a67eb0592a5f39a610a2e716840420" Jul 06 20:31:33.410481 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.410462 2583 scope.go:117] "RemoveContainer" containerID="2f28a582263e623d607c8c7eaeba18fbe6a67eb0592a5f39a610a2e716840420" Jul 06 20:31:33.410766 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:31:33.410748 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f28a582263e623d607c8c7eaeba18fbe6a67eb0592a5f39a610a2e716840420\": container with ID starting with 2f28a582263e623d607c8c7eaeba18fbe6a67eb0592a5f39a610a2e716840420 not found: ID does not exist" containerID="2f28a582263e623d607c8c7eaeba18fbe6a67eb0592a5f39a610a2e716840420" Jul 06 20:31:33.410830 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.410773 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f28a582263e623d607c8c7eaeba18fbe6a67eb0592a5f39a610a2e716840420"} err="failed to get container status \"2f28a582263e623d607c8c7eaeba18fbe6a67eb0592a5f39a610a2e716840420\": rpc error: code = NotFound desc = could not find container \"2f28a582263e623d607c8c7eaeba18fbe6a67eb0592a5f39a610a2e716840420\": container with ID starting with 2f28a582263e623d607c8c7eaeba18fbe6a67eb0592a5f39a610a2e716840420 not found: ID does not exist" Jul 06 20:31:33.430392 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.430363 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m"] Jul 06 20:31:33.438470 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.438450 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8a8b3310-openshift-default-895849675-ngs7m"] Jul 06 20:31:33.485000 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.484977 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" path="/var/lib/kubelet/pods/77c8ee68-97b1-4a7f-8c94-fefcc0b8d349/volumes" Jul 06 20:31:33.626129 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.626105 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44"] Jul 06 20:31:33.626490 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.626478 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" containerName="istio-proxy" Jul 06 20:31:33.626544 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.626492 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" containerName="istio-proxy" Jul 06 20:31:33.626544 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.626512 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8e28657-8eea-41ff-adce-53cf00ba48d2" containerName="cleanup" Jul 06 20:31:33.626544 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.626518 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8e28657-8eea-41ff-adce-53cf00ba48d2" containerName="cleanup" Jul 06 20:31:33.626640 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.626587 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8e28657-8eea-41ff-adce-53cf00ba48d2" containerName="cleanup" Jul 06 20:31:33.626640 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.626595 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="77c8ee68-97b1-4a7f-8c94-fefcc0b8d349" containerName="istio-proxy" Jul 06 20:31:33.629853 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.629832 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.632937 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.632918 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-915683a6-openshift-default-dockercfg-7bdj9\"" Jul 06 20:31:33.658174 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.658109 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44"] Jul 06 20:31:33.713803 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.713780 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-envoy\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.714174 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.713814 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istiod-ca-cert\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.714174 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.713840 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-workload-certs\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.714174 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.713941 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrqc4\" (UniqueName: \"kubernetes.io/projected/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-kube-api-access-zrqc4\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.714174 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.713974 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-token\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.714174 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.714003 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-credential-socket\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.714174 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.714101 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-podinfo\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.714174 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.714136 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-data\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.714174 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.714170 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-workload-socket\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.815095 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.815068 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-workload-certs\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.815251 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.815113 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrqc4\" (UniqueName: \"kubernetes.io/projected/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-kube-api-access-zrqc4\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.815251 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.815146 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-token\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.815251 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.815186 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-credential-socket\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.815251 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.815248 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-podinfo\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.815490 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.815274 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-data\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.815490 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.815306 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-workload-socket\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.815490 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.815356 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-envoy\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.815490 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.815393 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istiod-ca-cert\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.815684 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.815552 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-workload-certs\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.815684 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.815618 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-credential-socket\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.815684 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.815653 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-data\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.815784 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.815719 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-workload-socket\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.816127 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.816111 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istiod-ca-cert\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.817547 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.817528 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-envoy\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.817701 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.817684 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-podinfo\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.829706 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.829686 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-token\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.830845 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.830824 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrqc4\" (UniqueName: \"kubernetes.io/projected/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-kube-api-access-zrqc4\") pod \"e2e-ait-915683a6-openshift-default-69c69bcb9-47z44\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:33.941114 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:33.941048 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:34.305740 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:34.305717 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44"] Jul 06 20:31:34.306948 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:31:34.306923 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30763375_b8c7_4ddc_8a5e_adfc79f5f7ef.slice/crio-4b044d8877a1aaf35b0af94b30f6745f52cce3e4ccae0cdd933860c2930d376f WatchSource:0}: Error finding container 4b044d8877a1aaf35b0af94b30f6745f52cce3e4ccae0cdd933860c2930d376f: Status 404 returned error can't find the container with id 4b044d8877a1aaf35b0af94b30f6745f52cce3e4ccae0cdd933860c2930d376f Jul 06 20:31:34.308944 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:34.308910 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:31:34.309032 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:34.308990 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:31:34.309104 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:34.309037 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:31:34.409800 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:34.409769 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" event={"ID":"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef","Type":"ContainerStarted","Data":"aaec4c5494bd50a50a35e2e74868d4aca71ac574a9c54b25058d44dcbc87a702"} Jul 06 20:31:34.409943 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:34.409822 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" event={"ID":"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef","Type":"ContainerStarted","Data":"4b044d8877a1aaf35b0af94b30f6745f52cce3e4ccae0cdd933860c2930d376f"} Jul 06 20:31:34.435994 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:34.435945 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" podStartSLOduration=1.435929861 podStartE2EDuration="1.435929861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:31:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:31:34.433787794 +0000 UTC m=+1547.560699899" watchObservedRunningTime="2026-07-06 20:31:34.435929861 +0000 UTC m=+1547.562841964" Jul 06 20:31:34.941525 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:34.941488 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:35.946291 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:35.946262 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:36.422228 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:36.422195 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:36.423097 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:36.423077 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:46.000085 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:46.000053 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44"] Jul 06 20:31:46.000500 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:46.000232 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" podUID="30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" containerName="istio-proxy" containerID="cri-o://aaec4c5494bd50a50a35e2e74868d4aca71ac574a9c54b25058d44dcbc87a702" gracePeriod=30 Jul 06 20:31:51.259952 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.259885 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:51.361384 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.361350 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-token\") pod \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " Jul 06 20:31:51.361526 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.361392 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-podinfo\") pod \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " Jul 06 20:31:51.361526 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.361421 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrqc4\" (UniqueName: \"kubernetes.io/projected/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-kube-api-access-zrqc4\") pod \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " Jul 06 20:31:51.361526 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.361448 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-data\") pod \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " Jul 06 20:31:51.361655 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.361542 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-envoy\") pod \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " Jul 06 20:31:51.361655 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.361573 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-credential-socket\") pod \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " Jul 06 20:31:51.361655 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.361634 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-workload-certs\") pod \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " Jul 06 20:31:51.361817 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.361665 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-workload-socket\") pod \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " Jul 06 20:31:51.361817 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.361724 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istiod-ca-cert\") pod \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\" (UID: \"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef\") " Jul 06 20:31:51.361817 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.361753 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-data" (OuterVolumeSpecName: "istio-data") pod "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" (UID: "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:31:51.362007 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.361911 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" (UID: "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:31:51.362066 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.362017 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" (UID: "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:31:51.362174 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.362155 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:51.362248 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.362176 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:51.362248 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.362190 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:51.362248 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.362232 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" (UID: "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:31:51.362360 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.362231 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" (UID: "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:31:51.363708 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.363686 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-kube-api-access-zrqc4" (OuterVolumeSpecName: "kube-api-access-zrqc4") pod "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" (UID: "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef"). InnerVolumeSpecName "kube-api-access-zrqc4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:31:51.363769 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.363688 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" (UID: "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:31:51.363996 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.363974 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" (UID: "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:31:51.364064 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.363992 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-token" (OuterVolumeSpecName: "istio-token") pod "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" (UID: "30763375-b8c7-4ddc-8a5e-adfc79f5f7ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:31:51.463541 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.463516 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:51.463541 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.463541 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:51.463693 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.463549 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:51.463693 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.463557 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrqc4\" (UniqueName: \"kubernetes.io/projected/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-kube-api-access-zrqc4\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:51.463693 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.463566 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:51.463693 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.463574 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:31:51.516017 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.515952 2583 generic.go:356] "Generic (PLEG): container finished" podID="30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" containerID="aaec4c5494bd50a50a35e2e74868d4aca71ac574a9c54b25058d44dcbc87a702" exitCode=0 Jul 06 20:31:51.516017 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.516000 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" Jul 06 20:31:51.516171 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.516042 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" event={"ID":"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef","Type":"ContainerDied","Data":"aaec4c5494bd50a50a35e2e74868d4aca71ac574a9c54b25058d44dcbc87a702"} Jul 06 20:31:51.516171 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.516094 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44" event={"ID":"30763375-b8c7-4ddc-8a5e-adfc79f5f7ef","Type":"ContainerDied","Data":"4b044d8877a1aaf35b0af94b30f6745f52cce3e4ccae0cdd933860c2930d376f"} Jul 06 20:31:51.516171 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.516123 2583 scope.go:117] "RemoveContainer" containerID="aaec4c5494bd50a50a35e2e74868d4aca71ac574a9c54b25058d44dcbc87a702" Jul 06 20:31:51.525278 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.525262 2583 scope.go:117] "RemoveContainer" containerID="aaec4c5494bd50a50a35e2e74868d4aca71ac574a9c54b25058d44dcbc87a702" Jul 06 20:31:51.525524 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:31:51.525505 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaec4c5494bd50a50a35e2e74868d4aca71ac574a9c54b25058d44dcbc87a702\": container with ID starting with aaec4c5494bd50a50a35e2e74868d4aca71ac574a9c54b25058d44dcbc87a702 not found: ID does not exist" containerID="aaec4c5494bd50a50a35e2e74868d4aca71ac574a9c54b25058d44dcbc87a702" Jul 06 20:31:51.525593 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.525535 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaec4c5494bd50a50a35e2e74868d4aca71ac574a9c54b25058d44dcbc87a702"} err="failed to get container status \"aaec4c5494bd50a50a35e2e74868d4aca71ac574a9c54b25058d44dcbc87a702\": rpc error: code = NotFound desc = could not find container \"aaec4c5494bd50a50a35e2e74868d4aca71ac574a9c54b25058d44dcbc87a702\": container with ID starting with aaec4c5494bd50a50a35e2e74868d4aca71ac574a9c54b25058d44dcbc87a702 not found: ID does not exist" Jul 06 20:31:51.571953 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.571924 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44"] Jul 06 20:31:51.580373 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:51.580352 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-915683a6-openshift-default-69c69bcb9-47z44"] Jul 06 20:31:53.486308 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:53.486275 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" path="/var/lib/kubelet/pods/30763375-b8c7-4ddc-8a5e-adfc79f5f7ef/volumes" Jul 06 20:31:54.111397 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.111368 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7"] Jul 06 20:31:54.111797 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.111783 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" containerName="istio-proxy" Jul 06 20:31:54.111797 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.111797 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" containerName="istio-proxy" Jul 06 20:31:54.111946 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.111886 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="30763375-b8c7-4ddc-8a5e-adfc79f5f7ef" containerName="istio-proxy" Jul 06 20:31:54.116487 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.116466 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.123357 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.123337 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-2c6e75a8-openshift-default-dockercfg-bq98t\"" Jul 06 20:31:54.149674 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.149650 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7"] Jul 06 20:31:54.287234 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.287204 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-workload-certs\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.287379 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.287241 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-token\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.287379 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.287265 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-data\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.287379 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.287309 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-envoy\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.287379 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.287340 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istiod-ca-cert\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.287568 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.287396 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-podinfo\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.287568 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.287428 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-credential-socket\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.287568 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.287451 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-workload-socket\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.287568 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.287526 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j5xt\" (UniqueName: \"kubernetes.io/projected/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-kube-api-access-2j5xt\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.388591 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.388519 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-podinfo\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.388591 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.388550 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-credential-socket\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.388591 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.388573 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-workload-socket\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.388804 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.388617 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j5xt\" (UniqueName: \"kubernetes.io/projected/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-kube-api-access-2j5xt\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.388804 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.388692 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-workload-certs\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.388804 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.388718 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-token\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.388804 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.388745 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-data\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.388804 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.388775 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-envoy\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.389105 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.389018 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-credential-socket\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.389105 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.389038 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-workload-socket\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.389105 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.389095 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-data\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.389251 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.389129 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-workload-certs\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.389251 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.389213 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istiod-ca-cert\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.389743 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.389727 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istiod-ca-cert\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.391081 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.391059 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-envoy\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.391183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.391118 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-podinfo\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.399500 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.399479 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j5xt\" (UniqueName: \"kubernetes.io/projected/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-kube-api-access-2j5xt\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.400800 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.400781 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-token\") pod \"e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.426728 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.426712 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:54.584312 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.584288 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7"] Jul 06 20:31:54.586199 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:31:54.586168 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe42d75f_3d3e_4034_9daf_13c1cda1e9df.slice/crio-8123b574cc73b062e575affcd91485b70deda86312b776bfb94435714648c7e4 WatchSource:0}: Error finding container 8123b574cc73b062e575affcd91485b70deda86312b776bfb94435714648c7e4: Status 404 returned error can't find the container with id 8123b574cc73b062e575affcd91485b70deda86312b776bfb94435714648c7e4 Jul 06 20:31:54.588446 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.588416 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:31:54.588513 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.588478 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:31:54.588513 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:54.588505 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:31:55.546204 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:55.546170 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" event={"ID":"fe42d75f-3d3e-4034-9daf-13c1cda1e9df","Type":"ContainerStarted","Data":"e2dcb4916d877a2df538055e4288d9500d4c98cb4c53c89691481ef8f27d1b03"} Jul 06 20:31:55.546381 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:55.546220 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" event={"ID":"fe42d75f-3d3e-4034-9daf-13c1cda1e9df","Type":"ContainerStarted","Data":"8123b574cc73b062e575affcd91485b70deda86312b776bfb94435714648c7e4"} Jul 06 20:31:55.580322 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:55.580275 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" podStartSLOduration=1.5802590699999999 podStartE2EDuration="1.58025907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:31:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:31:55.576920059 +0000 UTC m=+1568.703832166" watchObservedRunningTime="2026-07-06 20:31:55.58025907 +0000 UTC m=+1568.707171174" Jul 06 20:31:56.427371 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:56.427338 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:56.432096 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:56.432070 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:56.552387 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:56.552356 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:56.553261 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:56.553236 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:31:59.937348 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:59.937311 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7"] Jul 06 20:31:59.937828 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:31:59.937499 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" podUID="fe42d75f-3d3e-4034-9daf-13c1cda1e9df" containerName="istio-proxy" containerID="cri-o://e2dcb4916d877a2df538055e4288d9500d4c98cb4c53c89691481ef8f27d1b03" gracePeriod=30 Jul 06 20:32:05.183346 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.183321 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:32:05.275785 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.275760 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-token\") pod \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " Jul 06 20:32:05.275943 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.275821 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-workload-certs\") pod \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " Jul 06 20:32:05.275943 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.275837 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-envoy\") pod \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " Jul 06 20:32:05.275943 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.275863 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-data\") pod \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " Jul 06 20:32:05.275943 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.275901 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-workload-socket\") pod \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " Jul 06 20:32:05.276185 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.275946 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-podinfo\") pod \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " Jul 06 20:32:05.276185 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.275966 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2j5xt\" (UniqueName: \"kubernetes.io/projected/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-kube-api-access-2j5xt\") pod \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " Jul 06 20:32:05.276185 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.276096 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-credential-socket\") pod \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " Jul 06 20:32:05.276185 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.276181 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istiod-ca-cert\") pod \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\" (UID: \"fe42d75f-3d3e-4034-9daf-13c1cda1e9df\") " Jul 06 20:32:05.276432 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.276098 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fe42d75f-3d3e-4034-9daf-13c1cda1e9df" (UID: "fe42d75f-3d3e-4034-9daf-13c1cda1e9df"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:32:05.276432 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.276267 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-data" (OuterVolumeSpecName: "istio-data") pod "fe42d75f-3d3e-4034-9daf-13c1cda1e9df" (UID: "fe42d75f-3d3e-4034-9daf-13c1cda1e9df"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:32:05.276432 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.276328 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fe42d75f-3d3e-4034-9daf-13c1cda1e9df" (UID: "fe42d75f-3d3e-4034-9daf-13c1cda1e9df"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:32:05.276432 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.276342 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fe42d75f-3d3e-4034-9daf-13c1cda1e9df" (UID: "fe42d75f-3d3e-4034-9daf-13c1cda1e9df"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:32:05.276636 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.276584 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fe42d75f-3d3e-4034-9daf-13c1cda1e9df" (UID: "fe42d75f-3d3e-4034-9daf-13c1cda1e9df"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:32:05.276636 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.276588 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:32:05.276743 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.276635 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:32:05.276743 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.276654 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:32:05.276743 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.276668 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:32:05.278093 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.278062 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fe42d75f-3d3e-4034-9daf-13c1cda1e9df" (UID: "fe42d75f-3d3e-4034-9daf-13c1cda1e9df"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:32:05.278093 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.278084 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-token" (OuterVolumeSpecName: "istio-token") pod "fe42d75f-3d3e-4034-9daf-13c1cda1e9df" (UID: "fe42d75f-3d3e-4034-9daf-13c1cda1e9df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:32:05.278235 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.278135 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fe42d75f-3d3e-4034-9daf-13c1cda1e9df" (UID: "fe42d75f-3d3e-4034-9daf-13c1cda1e9df"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:32:05.278235 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.278211 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-kube-api-access-2j5xt" (OuterVolumeSpecName: "kube-api-access-2j5xt") pod "fe42d75f-3d3e-4034-9daf-13c1cda1e9df" (UID: "fe42d75f-3d3e-4034-9daf-13c1cda1e9df"). InnerVolumeSpecName "kube-api-access-2j5xt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:32:05.377491 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.377465 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:32:05.377491 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.377489 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:32:05.377635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.377503 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2j5xt\" (UniqueName: \"kubernetes.io/projected/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-kube-api-access-2j5xt\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:32:05.377635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.377516 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:32:05.377635 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.377528 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe42d75f-3d3e-4034-9daf-13c1cda1e9df-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:32:05.609678 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.609604 2583 generic.go:356] "Generic (PLEG): container finished" podID="fe42d75f-3d3e-4034-9daf-13c1cda1e9df" containerID="e2dcb4916d877a2df538055e4288d9500d4c98cb4c53c89691481ef8f27d1b03" exitCode=0 Jul 06 20:32:05.609678 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.609648 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" Jul 06 20:32:05.609855 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.609674 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" event={"ID":"fe42d75f-3d3e-4034-9daf-13c1cda1e9df","Type":"ContainerDied","Data":"e2dcb4916d877a2df538055e4288d9500d4c98cb4c53c89691481ef8f27d1b03"} Jul 06 20:32:05.609855 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.609723 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7" event={"ID":"fe42d75f-3d3e-4034-9daf-13c1cda1e9df","Type":"ContainerDied","Data":"8123b574cc73b062e575affcd91485b70deda86312b776bfb94435714648c7e4"} Jul 06 20:32:05.609855 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.609754 2583 scope.go:117] "RemoveContainer" containerID="e2dcb4916d877a2df538055e4288d9500d4c98cb4c53c89691481ef8f27d1b03" Jul 06 20:32:05.618826 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.618809 2583 scope.go:117] "RemoveContainer" containerID="e2dcb4916d877a2df538055e4288d9500d4c98cb4c53c89691481ef8f27d1b03" Jul 06 20:32:05.619120 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:32:05.619100 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2dcb4916d877a2df538055e4288d9500d4c98cb4c53c89691481ef8f27d1b03\": container with ID starting with e2dcb4916d877a2df538055e4288d9500d4c98cb4c53c89691481ef8f27d1b03 not found: ID does not exist" containerID="e2dcb4916d877a2df538055e4288d9500d4c98cb4c53c89691481ef8f27d1b03" Jul 06 20:32:05.619225 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.619128 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2dcb4916d877a2df538055e4288d9500d4c98cb4c53c89691481ef8f27d1b03"} err="failed to get container status \"e2dcb4916d877a2df538055e4288d9500d4c98cb4c53c89691481ef8f27d1b03\": rpc error: code = NotFound desc = could not find container \"e2dcb4916d877a2df538055e4288d9500d4c98cb4c53c89691481ef8f27d1b03\": container with ID starting with e2dcb4916d877a2df538055e4288d9500d4c98cb4c53c89691481ef8f27d1b03 not found: ID does not exist" Jul 06 20:32:05.630324 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.630298 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7"] Jul 06 20:32:05.635193 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:05.635173 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-2c6e75a8-openshift-default-855cc55dff-gq5b7"] Jul 06 20:32:07.491026 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:32:07.490993 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe42d75f-3d3e-4034-9daf-13c1cda1e9df" path="/var/lib/kubelet/pods/fe42d75f-3d3e-4034-9daf-13c1cda1e9df/volumes" Jul 06 20:33:32.491968 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:32.491932 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:33:39.517293 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:39.517262 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:33:47.690923 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:47.690882 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:33:48.307786 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.307754 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss"] Jul 06 20:33:48.308284 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.308256 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe42d75f-3d3e-4034-9daf-13c1cda1e9df" containerName="istio-proxy" Jul 06 20:33:48.308284 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.308281 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe42d75f-3d3e-4034-9daf-13c1cda1e9df" containerName="istio-proxy" Jul 06 20:33:48.308451 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.308427 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe42d75f-3d3e-4034-9daf-13c1cda1e9df" containerName="istio-proxy" Jul 06 20:33:48.311599 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.311582 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.314795 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.314777 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a8dc09e7-openshift-default-dockercfg-r6jrc\"" Jul 06 20:33:48.385864 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.385838 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss"] Jul 06 20:33:48.410773 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.410750 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-workload-certs\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.410904 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.410789 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1603635-0086-4407-9d57-3d1897445886-istio-token\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.410904 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.410807 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-istio-data\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.410993 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.410906 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1603635-0086-4407-9d57-3d1897445886-istiod-ca-cert\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.410993 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.410938 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1603635-0086-4407-9d57-3d1897445886-istio-podinfo\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.411085 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.410993 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-istio-envoy\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.411085 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.411028 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-credential-socket\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.411085 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.411071 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rpxx\" (UniqueName: \"kubernetes.io/projected/f1603635-0086-4407-9d57-3d1897445886-kube-api-access-7rpxx\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.411239 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.411119 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-workload-socket\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.512037 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.512011 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-workload-socket\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.512193 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.512046 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-workload-certs\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.512193 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.512084 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1603635-0086-4407-9d57-3d1897445886-istio-token\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.512293 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.512200 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-istio-data\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.512293 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.512262 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1603635-0086-4407-9d57-3d1897445886-istiod-ca-cert\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.512411 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.512294 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1603635-0086-4407-9d57-3d1897445886-istio-podinfo\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.512411 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.512335 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-istio-envoy\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.512411 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.512361 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-credential-socket\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.512411 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.512394 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rpxx\" (UniqueName: \"kubernetes.io/projected/f1603635-0086-4407-9d57-3d1897445886-kube-api-access-7rpxx\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.512411 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.512399 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-workload-certs\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.512750 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.512451 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-workload-socket\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.512750 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.512627 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-istio-data\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.512847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.512746 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-credential-socket\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.513071 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.513044 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1603635-0086-4407-9d57-3d1897445886-istiod-ca-cert\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.514633 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.514609 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-istio-envoy\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.514809 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.514791 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1603635-0086-4407-9d57-3d1897445886-istio-podinfo\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.522056 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.522034 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1603635-0086-4407-9d57-3d1897445886-istio-token\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.522220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.522198 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rpxx\" (UniqueName: \"kubernetes.io/projected/f1603635-0086-4407-9d57-3d1897445886-kube-api-access-7rpxx\") pod \"e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.622294 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.622235 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:48.770991 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.770962 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss"] Jul 06 20:33:48.774088 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:33:48.774061 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1603635_0086_4407_9d57_3d1897445886.slice/crio-1e906644018aafb4348ab1ee6b2cd1c6858f406b0a3ff3a498f4d6ec2af94429 WatchSource:0}: Error finding container 1e906644018aafb4348ab1ee6b2cd1c6858f406b0a3ff3a498f4d6ec2af94429: Status 404 returned error can't find the container with id 1e906644018aafb4348ab1ee6b2cd1c6858f406b0a3ff3a498f4d6ec2af94429 Jul 06 20:33:48.776088 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.776057 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:33:48.776184 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.776117 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:33:48.776184 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:48.776145 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:33:49.235797 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:49.235766 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" event={"ID":"f1603635-0086-4407-9d57-3d1897445886","Type":"ContainerStarted","Data":"ad9370a931cc0e9aa8accdfd61323f7c2658cb6885803047b6ad7e3d481d07d4"} Jul 06 20:33:49.236009 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:49.235814 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" event={"ID":"f1603635-0086-4407-9d57-3d1897445886","Type":"ContainerStarted","Data":"1e906644018aafb4348ab1ee6b2cd1c6858f406b0a3ff3a498f4d6ec2af94429"} Jul 06 20:33:49.258817 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:49.258774 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" podStartSLOduration=1.258758753 podStartE2EDuration="1.258758753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:33:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:33:49.257253605 +0000 UTC m=+1682.384165709" watchObservedRunningTime="2026-07-06 20:33:49.258758753 +0000 UTC m=+1682.385670857" Jul 06 20:33:49.622573 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:49.622494 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:49.627432 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:49.627406 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:50.241749 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:50.241718 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:50.242772 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:50.242751 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:33:56.715848 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:33:56.715811 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:34:05.243819 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:05.243788 2583 patch_prober.go:28] interesting pod/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 06 20:34:05.244243 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:05.243886 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" podUID="f1603635-0086-4407-9d57-3d1897445886" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 06 20:34:05.337274 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:05.337242 2583 generic.go:356] "Generic (PLEG): container finished" podID="f1603635-0086-4407-9d57-3d1897445886" containerID="ad9370a931cc0e9aa8accdfd61323f7c2658cb6885803047b6ad7e3d481d07d4" exitCode=0 Jul 06 20:34:05.337416 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:05.337282 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" event={"ID":"f1603635-0086-4407-9d57-3d1897445886","Type":"ContainerDied","Data":"ad9370a931cc0e9aa8accdfd61323f7c2658cb6885803047b6ad7e3d481d07d4"} Jul 06 20:34:05.337618 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:05.337606 2583 scope.go:117] "RemoveContainer" containerID="ad9370a931cc0e9aa8accdfd61323f7c2658cb6885803047b6ad7e3d481d07d4" Jul 06 20:34:05.338662 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:05.338631 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:34:05.338719 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:05.338699 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:34:05.338755 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:05.338730 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:34:06.351510 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:06.351465 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" event={"ID":"f1603635-0086-4407-9d57-3d1897445886","Type":"ContainerStarted","Data":"04cf39cf10c5b31c0bc8589a24a01784a02813656eb283e54877268448ee794a"} Jul 06 20:34:06.622494 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:06.622412 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:34:06.622648 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:06.622589 2583 patch_prober.go:28] interesting pod/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 06 20:34:06.622691 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:06.622645 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" podUID="f1603635-0086-4407-9d57-3d1897445886" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 06 20:34:07.623224 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:07.623192 2583 patch_prober.go:28] interesting pod/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 06 20:34:07.623619 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:07.623247 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" podUID="f1603635-0086-4407-9d57-3d1897445886" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 06 20:34:07.705164 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:07.705130 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:34:08.623608 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:08.623574 2583 patch_prober.go:28] interesting pod/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 06 20:34:08.624005 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:08.623624 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" podUID="f1603635-0086-4407-9d57-3d1897445886" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 06 20:34:09.623123 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:09.623035 2583 patch_prober.go:28] interesting pod/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 06 20:34:09.623123 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:09.623084 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" podUID="f1603635-0086-4407-9d57-3d1897445886" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 06 20:34:10.626740 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:10.626709 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:34:10.627256 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:10.627236 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:34:10.627961 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:10.627937 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:34:11.107225 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:11.107194 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss"] Jul 06 20:34:11.381513 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:11.381433 2583 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" secret="" err="secret \"e2e-mt-a8dc09e7-openshift-default-dockercfg-r6jrc\" not found" Jul 06 20:34:12.387716 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:12.387677 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" podUID="f1603635-0086-4407-9d57-3d1897445886" containerName="istio-proxy" containerID="cri-o://04cf39cf10c5b31c0bc8589a24a01784a02813656eb283e54877268448ee794a" gracePeriod=30 Jul 06 20:34:17.644391 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.644368 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:34:17.783146 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.783119 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-credential-socket\") pod \"f1603635-0086-4407-9d57-3d1897445886\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " Jul 06 20:34:17.783276 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.783171 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1603635-0086-4407-9d57-3d1897445886-istio-token\") pod \"f1603635-0086-4407-9d57-3d1897445886\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " Jul 06 20:34:17.783276 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.783194 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-istio-envoy\") pod \"f1603635-0086-4407-9d57-3d1897445886\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " Jul 06 20:34:17.783276 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.783217 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-istio-data\") pod \"f1603635-0086-4407-9d57-3d1897445886\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " Jul 06 20:34:17.783276 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.783233 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rpxx\" (UniqueName: \"kubernetes.io/projected/f1603635-0086-4407-9d57-3d1897445886-kube-api-access-7rpxx\") pod \"f1603635-0086-4407-9d57-3d1897445886\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " Jul 06 20:34:17.783276 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.783269 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1603635-0086-4407-9d57-3d1897445886-istiod-ca-cert\") pod \"f1603635-0086-4407-9d57-3d1897445886\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " Jul 06 20:34:17.783552 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.783323 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-workload-certs\") pod \"f1603635-0086-4407-9d57-3d1897445886\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " Jul 06 20:34:17.783552 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.783347 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-workload-socket\") pod \"f1603635-0086-4407-9d57-3d1897445886\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " Jul 06 20:34:17.783552 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.783377 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f1603635-0086-4407-9d57-3d1897445886" (UID: "f1603635-0086-4407-9d57-3d1897445886"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:34:17.783552 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.783395 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1603635-0086-4407-9d57-3d1897445886-istio-podinfo\") pod \"f1603635-0086-4407-9d57-3d1897445886\" (UID: \"f1603635-0086-4407-9d57-3d1897445886\") " Jul 06 20:34:17.784156 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.783741 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f1603635-0086-4407-9d57-3d1897445886" (UID: "f1603635-0086-4407-9d57-3d1897445886"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:34:17.784156 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.783748 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1603635-0086-4407-9d57-3d1897445886-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f1603635-0086-4407-9d57-3d1897445886" (UID: "f1603635-0086-4407-9d57-3d1897445886"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:34:17.784156 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.783766 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:34:17.784156 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.783953 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f1603635-0086-4407-9d57-3d1897445886" (UID: "f1603635-0086-4407-9d57-3d1897445886"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:34:17.784416 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.784397 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-istio-data" (OuterVolumeSpecName: "istio-data") pod "f1603635-0086-4407-9d57-3d1897445886" (UID: "f1603635-0086-4407-9d57-3d1897445886"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:34:17.785924 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.785868 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f1603635-0086-4407-9d57-3d1897445886" (UID: "f1603635-0086-4407-9d57-3d1897445886"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:34:17.785924 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.785896 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1603635-0086-4407-9d57-3d1897445886-istio-token" (OuterVolumeSpecName: "istio-token") pod "f1603635-0086-4407-9d57-3d1897445886" (UID: "f1603635-0086-4407-9d57-3d1897445886"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:34:17.786060 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.785919 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f1603635-0086-4407-9d57-3d1897445886-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f1603635-0086-4407-9d57-3d1897445886" (UID: "f1603635-0086-4407-9d57-3d1897445886"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:34:17.786060 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.785930 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1603635-0086-4407-9d57-3d1897445886-kube-api-access-7rpxx" (OuterVolumeSpecName: "kube-api-access-7rpxx") pod "f1603635-0086-4407-9d57-3d1897445886" (UID: "f1603635-0086-4407-9d57-3d1897445886"). InnerVolumeSpecName "kube-api-access-7rpxx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:34:17.885036 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.885008 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1603635-0086-4407-9d57-3d1897445886-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:34:17.885036 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.885037 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:34:17.885036 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.885045 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:34:17.885036 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.885054 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rpxx\" (UniqueName: \"kubernetes.io/projected/f1603635-0086-4407-9d57-3d1897445886-kube-api-access-7rpxx\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:34:17.885236 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.885063 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1603635-0086-4407-9d57-3d1897445886-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:34:17.885236 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.885071 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:34:17.885236 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.885078 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1603635-0086-4407-9d57-3d1897445886-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:34:17.885236 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:17.885088 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1603635-0086-4407-9d57-3d1897445886-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:34:18.425123 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:18.425092 2583 generic.go:356] "Generic (PLEG): container finished" podID="f1603635-0086-4407-9d57-3d1897445886" containerID="04cf39cf10c5b31c0bc8589a24a01784a02813656eb283e54877268448ee794a" exitCode=0 Jul 06 20:34:18.425273 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:18.425135 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" Jul 06 20:34:18.425273 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:18.425173 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" event={"ID":"f1603635-0086-4407-9d57-3d1897445886","Type":"ContainerDied","Data":"04cf39cf10c5b31c0bc8589a24a01784a02813656eb283e54877268448ee794a"} Jul 06 20:34:18.425273 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:18.425230 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss" event={"ID":"f1603635-0086-4407-9d57-3d1897445886","Type":"ContainerDied","Data":"1e906644018aafb4348ab1ee6b2cd1c6858f406b0a3ff3a498f4d6ec2af94429"} Jul 06 20:34:18.425273 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:18.425260 2583 scope.go:117] "RemoveContainer" containerID="04cf39cf10c5b31c0bc8589a24a01784a02813656eb283e54877268448ee794a" Jul 06 20:34:18.436338 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:18.436317 2583 scope.go:117] "RemoveContainer" containerID="ad9370a931cc0e9aa8accdfd61323f7c2658cb6885803047b6ad7e3d481d07d4" Jul 06 20:34:18.446774 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:18.446751 2583 scope.go:117] "RemoveContainer" containerID="04cf39cf10c5b31c0bc8589a24a01784a02813656eb283e54877268448ee794a" Jul 06 20:34:18.447061 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:34:18.447042 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04cf39cf10c5b31c0bc8589a24a01784a02813656eb283e54877268448ee794a\": container with ID starting with 04cf39cf10c5b31c0bc8589a24a01784a02813656eb283e54877268448ee794a not found: ID does not exist" containerID="04cf39cf10c5b31c0bc8589a24a01784a02813656eb283e54877268448ee794a" Jul 06 20:34:18.447120 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:18.447071 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04cf39cf10c5b31c0bc8589a24a01784a02813656eb283e54877268448ee794a"} err="failed to get container status \"04cf39cf10c5b31c0bc8589a24a01784a02813656eb283e54877268448ee794a\": rpc error: code = NotFound desc = could not find container \"04cf39cf10c5b31c0bc8589a24a01784a02813656eb283e54877268448ee794a\": container with ID starting with 04cf39cf10c5b31c0bc8589a24a01784a02813656eb283e54877268448ee794a not found: ID does not exist" Jul 06 20:34:18.447120 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:18.447087 2583 scope.go:117] "RemoveContainer" containerID="ad9370a931cc0e9aa8accdfd61323f7c2658cb6885803047b6ad7e3d481d07d4" Jul 06 20:34:18.447331 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:34:18.447312 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad9370a931cc0e9aa8accdfd61323f7c2658cb6885803047b6ad7e3d481d07d4\": container with ID starting with ad9370a931cc0e9aa8accdfd61323f7c2658cb6885803047b6ad7e3d481d07d4 not found: ID does not exist" containerID="ad9370a931cc0e9aa8accdfd61323f7c2658cb6885803047b6ad7e3d481d07d4" Jul 06 20:34:18.447383 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:18.447338 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad9370a931cc0e9aa8accdfd61323f7c2658cb6885803047b6ad7e3d481d07d4"} err="failed to get container status \"ad9370a931cc0e9aa8accdfd61323f7c2658cb6885803047b6ad7e3d481d07d4\": rpc error: code = NotFound desc = could not find container \"ad9370a931cc0e9aa8accdfd61323f7c2658cb6885803047b6ad7e3d481d07d4\": container with ID starting with ad9370a931cc0e9aa8accdfd61323f7c2658cb6885803047b6ad7e3d481d07d4 not found: ID does not exist" Jul 06 20:34:18.463794 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:18.463770 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss"] Jul 06 20:34:18.467811 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:18.467789 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a8dc09e7-openshift-default-74d6df7b79-8svss"] Jul 06 20:34:19.486276 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:19.486249 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1603635-0086-4407-9d57-3d1897445886" path="/var/lib/kubelet/pods/f1603635-0086-4407-9d57-3d1897445886/volumes" Jul 06 20:34:32.111542 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.111506 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg"] Jul 06 20:34:32.112193 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.112170 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1603635-0086-4407-9d57-3d1897445886" containerName="istio-proxy" Jul 06 20:34:32.112254 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.112198 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1603635-0086-4407-9d57-3d1897445886" containerName="istio-proxy" Jul 06 20:34:32.112354 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.112340 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1603635-0086-4407-9d57-3d1897445886" containerName="istio-proxy" Jul 06 20:34:32.112390 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.112360 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1603635-0086-4407-9d57-3d1897445886" containerName="istio-proxy" Jul 06 20:34:32.116129 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.116111 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.118554 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.118538 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-080e3c37-openshift-default-dockercfg-n6qrp\"" Jul 06 20:34:32.126502 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.126481 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg"] Jul 06 20:34:32.202193 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.202150 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-workload-certs\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.202399 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.202215 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istiod-ca-cert\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.202399 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.202256 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-podinfo\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.202399 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.202286 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-envoy\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.202399 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.202335 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-workload-socket\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.202399 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.202380 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-data\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.202690 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.202414 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-credential-socket\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.202690 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.202445 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-token\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.202690 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.202535 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsrr9\" (UniqueName: \"kubernetes.io/projected/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-kube-api-access-vsrr9\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.303779 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.303746 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-podinfo\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.303978 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.303796 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-envoy\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.303978 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.303830 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-workload-socket\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.303978 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.303859 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-data\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.303978 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.303907 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-credential-socket\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.303978 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.303938 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-token\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.304284 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.304027 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsrr9\" (UniqueName: \"kubernetes.io/projected/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-kube-api-access-vsrr9\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.304284 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.304093 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-workload-certs\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.304284 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.304131 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istiod-ca-cert\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.304441 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.304420 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-workload-socket\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.304852 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.304829 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istiod-ca-cert\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.305326 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.305268 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-data\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.305443 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.305404 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-credential-socket\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.305443 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.305423 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-workload-certs\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.307313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.307272 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-envoy\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.307567 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.307542 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-podinfo\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.318401 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.318376 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsrr9\" (UniqueName: \"kubernetes.io/projected/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-kube-api-access-vsrr9\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.318719 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.318700 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-token\") pod \"e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.428677 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.428585 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:32.609448 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.609413 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg"] Jul 06 20:34:32.611224 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:34:32.611198 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff9cda6_a8cc_4480_8314_f35eafa90ed3.slice/crio-6bf7fed9cb6ed6c379ee4f302e07bde99f0f34b0326c78bb03110f061d9ab4a7 WatchSource:0}: Error finding container 6bf7fed9cb6ed6c379ee4f302e07bde99f0f34b0326c78bb03110f061d9ab4a7: Status 404 returned error can't find the container with id 6bf7fed9cb6ed6c379ee4f302e07bde99f0f34b0326c78bb03110f061d9ab4a7 Jul 06 20:34:32.613353 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.613316 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:34:32.613440 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.613386 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:34:32.613440 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:32.613415 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:34:33.521433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:33.521401 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" event={"ID":"3ff9cda6-a8cc-4480-8314-f35eafa90ed3","Type":"ContainerStarted","Data":"ca77046b515f6f44b0600c531ae8b107a056ce5eb26eaed154d192a80cb42d82"} Jul 06 20:34:33.521809 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:33.521445 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" event={"ID":"3ff9cda6-a8cc-4480-8314-f35eafa90ed3","Type":"ContainerStarted","Data":"6bf7fed9cb6ed6c379ee4f302e07bde99f0f34b0326c78bb03110f061d9ab4a7"} Jul 06 20:34:33.547241 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:33.547183 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" podStartSLOduration=1.547164749 podStartE2EDuration="1.547164749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:34:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:34:33.543323272 +0000 UTC m=+1726.670235376" watchObservedRunningTime="2026-07-06 20:34:33.547164749 +0000 UTC m=+1726.674076854" Jul 06 20:34:34.429074 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:34.429042 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:34.434039 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:34.434018 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:34.528918 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:34.528891 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:34.529802 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:34.529784 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:39.696946 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:39.696912 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:34:47.609313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:47.609246 2583 generic.go:356] "Generic (PLEG): container finished" podID="3ff9cda6-a8cc-4480-8314-f35eafa90ed3" containerID="ca77046b515f6f44b0600c531ae8b107a056ce5eb26eaed154d192a80cb42d82" exitCode=0 Jul 06 20:34:47.609664 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:47.609316 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" event={"ID":"3ff9cda6-a8cc-4480-8314-f35eafa90ed3","Type":"ContainerDied","Data":"ca77046b515f6f44b0600c531ae8b107a056ce5eb26eaed154d192a80cb42d82"} Jul 06 20:34:47.609664 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:47.609628 2583 scope.go:117] "RemoveContainer" containerID="ca77046b515f6f44b0600c531ae8b107a056ce5eb26eaed154d192a80cb42d82" Jul 06 20:34:47.610543 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:47.610515 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:34:47.610641 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:47.610594 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:34:47.610708 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:47.610641 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:34:48.617011 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:48.616979 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" event={"ID":"3ff9cda6-a8cc-4480-8314-f35eafa90ed3","Type":"ContainerStarted","Data":"e7ee7b4adc8ed77c9309c801ec130e319eaf124745827a0d2440c330703eaca4"} Jul 06 20:34:49.429240 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:49.429201 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:49.430462 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:49.430427 2583 patch_prober.go:28] interesting pod/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 06 20:34:49.430597 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:49.430498 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" podUID="3ff9cda6-a8cc-4480-8314-f35eafa90ed3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 06 20:34:49.528979 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:49.528935 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:50.213882 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:50.213836 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:34:50.430897 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:50.430024 2583 patch_prober.go:28] interesting pod/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 06 20:34:50.430897 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:50.430077 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" podUID="3ff9cda6-a8cc-4480-8314-f35eafa90ed3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 06 20:34:51.429855 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:51.429815 2583 patch_prober.go:28] interesting pod/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 06 20:34:51.430349 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:51.429904 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" podUID="3ff9cda6-a8cc-4480-8314-f35eafa90ed3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 06 20:34:52.434170 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:52.434136 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:34:52.435016 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:34:52.434997 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:35:07.907316 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:35:07.907284 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:35:16.712041 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:35:16.712008 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:35:25.403111 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:35:25.403076 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:35:47.479572 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:35:47.479541 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:35:47.484360 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:35:47.484340 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:36:03.801934 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:03.801903 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:36:08.635115 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:08.635082 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg"] Jul 06 20:36:08.635579 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:08.635329 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" podUID="3ff9cda6-a8cc-4480-8314-f35eafa90ed3" containerName="istio-proxy" containerID="cri-o://e7ee7b4adc8ed77c9309c801ec130e319eaf124745827a0d2440c330703eaca4" gracePeriod=30 Jul 06 20:36:13.881816 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.881789 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:36:13.971712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.971642 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-workload-socket\") pod \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " Jul 06 20:36:13.971712 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.971675 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-token\") pod \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " Jul 06 20:36:13.971934 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.971725 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istiod-ca-cert\") pod \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " Jul 06 20:36:13.971934 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.971741 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-data\") pod \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " Jul 06 20:36:13.971934 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.971783 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-podinfo\") pod \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " Jul 06 20:36:13.971934 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.971810 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-envoy\") pod \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " Jul 06 20:36:13.971934 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.971859 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-workload-certs\") pod \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " Jul 06 20:36:13.971934 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.971923 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-credential-socket\") pod \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " Jul 06 20:36:13.972261 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.971945 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3ff9cda6-a8cc-4480-8314-f35eafa90ed3" (UID: "3ff9cda6-a8cc-4480-8314-f35eafa90ed3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:36:13.972261 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.971955 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsrr9\" (UniqueName: \"kubernetes.io/projected/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-kube-api-access-vsrr9\") pod \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\" (UID: \"3ff9cda6-a8cc-4480-8314-f35eafa90ed3\") " Jul 06 20:36:13.972261 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.972179 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3ff9cda6-a8cc-4480-8314-f35eafa90ed3" (UID: "3ff9cda6-a8cc-4480-8314-f35eafa90ed3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:36:13.972422 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.972400 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3ff9cda6-a8cc-4480-8314-f35eafa90ed3" (UID: "3ff9cda6-a8cc-4480-8314-f35eafa90ed3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:36:13.972484 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.972418 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3ff9cda6-a8cc-4480-8314-f35eafa90ed3" (UID: "3ff9cda6-a8cc-4480-8314-f35eafa90ed3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:36:13.972538 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.972512 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:36:13.972538 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.972529 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:36:13.972642 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.972544 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:36:13.972642 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.972557 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:36:13.972642 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.972630 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-data" (OuterVolumeSpecName: "istio-data") pod "3ff9cda6-a8cc-4480-8314-f35eafa90ed3" (UID: "3ff9cda6-a8cc-4480-8314-f35eafa90ed3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:36:13.974138 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.974112 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3ff9cda6-a8cc-4480-8314-f35eafa90ed3" (UID: "3ff9cda6-a8cc-4480-8314-f35eafa90ed3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:36:13.974253 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.974119 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3ff9cda6-a8cc-4480-8314-f35eafa90ed3" (UID: "3ff9cda6-a8cc-4480-8314-f35eafa90ed3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:36:13.974393 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.974375 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-kube-api-access-vsrr9" (OuterVolumeSpecName: "kube-api-access-vsrr9") pod "3ff9cda6-a8cc-4480-8314-f35eafa90ed3" (UID: "3ff9cda6-a8cc-4480-8314-f35eafa90ed3"). InnerVolumeSpecName "kube-api-access-vsrr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:36:13.974457 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:13.974387 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ff9cda6-a8cc-4480-8314-f35eafa90ed3" (UID: "3ff9cda6-a8cc-4480-8314-f35eafa90ed3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:36:14.073110 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.073089 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsrr9\" (UniqueName: \"kubernetes.io/projected/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-kube-api-access-vsrr9\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:36:14.073110 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.073110 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:36:14.073253 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.073120 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:36:14.073253 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.073128 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:36:14.073253 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.073135 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ff9cda6-a8cc-4480-8314-f35eafa90ed3-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:36:14.126143 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.126121 2583 generic.go:356] "Generic (PLEG): container finished" podID="3ff9cda6-a8cc-4480-8314-f35eafa90ed3" containerID="e7ee7b4adc8ed77c9309c801ec130e319eaf124745827a0d2440c330703eaca4" exitCode=0 Jul 06 20:36:14.126233 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.126168 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" Jul 06 20:36:14.126233 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.126195 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" event={"ID":"3ff9cda6-a8cc-4480-8314-f35eafa90ed3","Type":"ContainerDied","Data":"e7ee7b4adc8ed77c9309c801ec130e319eaf124745827a0d2440c330703eaca4"} Jul 06 20:36:14.126309 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.126243 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg" event={"ID":"3ff9cda6-a8cc-4480-8314-f35eafa90ed3","Type":"ContainerDied","Data":"6bf7fed9cb6ed6c379ee4f302e07bde99f0f34b0326c78bb03110f061d9ab4a7"} Jul 06 20:36:14.126309 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.126266 2583 scope.go:117] "RemoveContainer" containerID="e7ee7b4adc8ed77c9309c801ec130e319eaf124745827a0d2440c330703eaca4" Jul 06 20:36:14.143897 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.143855 2583 scope.go:117] "RemoveContainer" containerID="ca77046b515f6f44b0600c531ae8b107a056ce5eb26eaed154d192a80cb42d82" Jul 06 20:36:14.152769 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.152752 2583 scope.go:117] "RemoveContainer" containerID="e7ee7b4adc8ed77c9309c801ec130e319eaf124745827a0d2440c330703eaca4" Jul 06 20:36:14.153063 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:36:14.153044 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7ee7b4adc8ed77c9309c801ec130e319eaf124745827a0d2440c330703eaca4\": container with ID starting with e7ee7b4adc8ed77c9309c801ec130e319eaf124745827a0d2440c330703eaca4 not found: ID does not exist" containerID="e7ee7b4adc8ed77c9309c801ec130e319eaf124745827a0d2440c330703eaca4" Jul 06 20:36:14.153138 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.153073 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7ee7b4adc8ed77c9309c801ec130e319eaf124745827a0d2440c330703eaca4"} err="failed to get container status \"e7ee7b4adc8ed77c9309c801ec130e319eaf124745827a0d2440c330703eaca4\": rpc error: code = NotFound desc = could not find container \"e7ee7b4adc8ed77c9309c801ec130e319eaf124745827a0d2440c330703eaca4\": container with ID starting with e7ee7b4adc8ed77c9309c801ec130e319eaf124745827a0d2440c330703eaca4 not found: ID does not exist" Jul 06 20:36:14.153138 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.153096 2583 scope.go:117] "RemoveContainer" containerID="ca77046b515f6f44b0600c531ae8b107a056ce5eb26eaed154d192a80cb42d82" Jul 06 20:36:14.153338 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:36:14.153321 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca77046b515f6f44b0600c531ae8b107a056ce5eb26eaed154d192a80cb42d82\": container with ID starting with ca77046b515f6f44b0600c531ae8b107a056ce5eb26eaed154d192a80cb42d82 not found: ID does not exist" containerID="ca77046b515f6f44b0600c531ae8b107a056ce5eb26eaed154d192a80cb42d82" Jul 06 20:36:14.153391 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.153343 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca77046b515f6f44b0600c531ae8b107a056ce5eb26eaed154d192a80cb42d82"} err="failed to get container status \"ca77046b515f6f44b0600c531ae8b107a056ce5eb26eaed154d192a80cb42d82\": rpc error: code = NotFound desc = could not find container \"ca77046b515f6f44b0600c531ae8b107a056ce5eb26eaed154d192a80cb42d82\": container with ID starting with ca77046b515f6f44b0600c531ae8b107a056ce5eb26eaed154d192a80cb42d82 not found: ID does not exist" Jul 06 20:36:14.155376 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.155357 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg"] Jul 06 20:36:14.160308 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:14.160287 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-080e3c37-openshift-default-9f568765f-9rwpg"] Jul 06 20:36:15.485888 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:15.485847 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ff9cda6-a8cc-4480-8314-f35eafa90ed3" path="/var/lib/kubelet/pods/3ff9cda6-a8cc-4480-8314-f35eafa90ed3/volumes" Jul 06 20:36:16.387898 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:16.387852 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:36:27.695128 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:36:27.695095 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:37:05.866396 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.866362 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql"] Jul 06 20:37:05.867012 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.866980 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ff9cda6-a8cc-4480-8314-f35eafa90ed3" containerName="istio-proxy" Jul 06 20:37:05.867108 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.867015 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ff9cda6-a8cc-4480-8314-f35eafa90ed3" containerName="istio-proxy" Jul 06 20:37:05.867108 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.867063 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1603635-0086-4407-9d57-3d1897445886" containerName="istio-proxy" Jul 06 20:37:05.867108 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.867073 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1603635-0086-4407-9d57-3d1897445886" containerName="istio-proxy" Jul 06 20:37:05.867267 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.867176 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ff9cda6-a8cc-4480-8314-f35eafa90ed3" containerName="istio-proxy" Jul 06 20:37:05.867267 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.867191 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ff9cda6-a8cc-4480-8314-f35eafa90ed3" containerName="istio-proxy" Jul 06 20:37:05.870481 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.870461 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:05.877290 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.877271 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-cb2bb9-openshift-default-dockercfg-zfssj\"" Jul 06 20:37:05.896260 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.896236 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql"] Jul 06 20:37:05.997005 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.996976 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9000b594-3b6e-4044-b0e1-4532369efb03-istiod-ca-cert\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:05.997171 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.997014 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-istio-envoy\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:05.997171 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.997041 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-workload-certs\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:05.997171 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.997066 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-credential-socket\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:05.997171 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.997104 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7hs2\" (UniqueName: \"kubernetes.io/projected/9000b594-3b6e-4044-b0e1-4532369efb03-kube-api-access-x7hs2\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:05.997171 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.997132 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-workload-socket\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:05.997419 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.997179 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9000b594-3b6e-4044-b0e1-4532369efb03-istio-token\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:05.997419 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.997196 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-istio-data\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:05.997419 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:05.997309 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9000b594-3b6e-4044-b0e1-4532369efb03-istio-podinfo\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.098803 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.098745 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9000b594-3b6e-4044-b0e1-4532369efb03-istio-podinfo\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.099025 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.098822 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9000b594-3b6e-4044-b0e1-4532369efb03-istiod-ca-cert\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.099025 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.098860 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-istio-envoy\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.099025 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.098912 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-workload-certs\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.099264 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.099044 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-credential-socket\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.099264 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.099082 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7hs2\" (UniqueName: \"kubernetes.io/projected/9000b594-3b6e-4044-b0e1-4532369efb03-kube-api-access-x7hs2\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.099264 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.099118 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-workload-socket\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.099264 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.099185 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9000b594-3b6e-4044-b0e1-4532369efb03-istio-token\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.099514 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.099455 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-istio-data\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.099674 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.099646 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-credential-socket\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.099752 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.099713 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-workload-certs\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.099752 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.099740 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9000b594-3b6e-4044-b0e1-4532369efb03-istiod-ca-cert\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.099896 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.099781 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-workload-socket\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.099965 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.099898 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-istio-data\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.101573 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.101546 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9000b594-3b6e-4044-b0e1-4532369efb03-istio-podinfo\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.101855 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.101838 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-istio-envoy\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.112119 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.112084 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9000b594-3b6e-4044-b0e1-4532369efb03-istio-token\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.113962 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.113942 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7hs2\" (UniqueName: \"kubernetes.io/projected/9000b594-3b6e-4044-b0e1-4532369efb03-kube-api-access-x7hs2\") pod \"e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.180848 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.180781 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:06.333956 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.333927 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql"] Jul 06 20:37:06.335894 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:37:06.335844 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9000b594_3b6e_4044_b0e1_4532369efb03.slice/crio-a615e03655ea76d59953aa3bcbc13554f085568bf3743c913ada0ddbd71a7914 WatchSource:0}: Error finding container a615e03655ea76d59953aa3bcbc13554f085568bf3743c913ada0ddbd71a7914: Status 404 returned error can't find the container with id a615e03655ea76d59953aa3bcbc13554f085568bf3743c913ada0ddbd71a7914 Jul 06 20:37:06.338020 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.337989 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:37:06.338095 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.338052 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:37:06.338095 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.338081 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:37:06.439645 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.439493 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" event={"ID":"9000b594-3b6e-4044-b0e1-4532369efb03","Type":"ContainerStarted","Data":"ca0d7c87ff0c6bb2eff40a48c17c945a31a36f5350fa234ec8676cec9204417c"} Jul 06 20:37:06.439645 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.439557 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" event={"ID":"9000b594-3b6e-4044-b0e1-4532369efb03","Type":"ContainerStarted","Data":"a615e03655ea76d59953aa3bcbc13554f085568bf3743c913ada0ddbd71a7914"} Jul 06 20:37:06.463990 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:06.463937 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" podStartSLOduration=1.463917406 podStartE2EDuration="1.463917406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:37:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:37:06.462391861 +0000 UTC m=+1879.589303964" watchObservedRunningTime="2026-07-06 20:37:06.463917406 +0000 UTC m=+1879.590829509" Jul 06 20:37:07.181216 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:07.181183 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:07.186293 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:07.186261 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:07.445953 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:07.445863 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:07.446756 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:07.446740 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:12.496469 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:12.496389 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:37:20.523204 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:20.523178 2583 generic.go:356] "Generic (PLEG): container finished" podID="9000b594-3b6e-4044-b0e1-4532369efb03" containerID="ca0d7c87ff0c6bb2eff40a48c17c945a31a36f5350fa234ec8676cec9204417c" exitCode=0 Jul 06 20:37:20.523594 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:20.523249 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" event={"ID":"9000b594-3b6e-4044-b0e1-4532369efb03","Type":"ContainerDied","Data":"ca0d7c87ff0c6bb2eff40a48c17c945a31a36f5350fa234ec8676cec9204417c"} Jul 06 20:37:20.523677 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:20.523662 2583 scope.go:117] "RemoveContainer" containerID="ca0d7c87ff0c6bb2eff40a48c17c945a31a36f5350fa234ec8676cec9204417c" Jul 06 20:37:20.524726 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:20.524701 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:37:20.524785 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:20.524765 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:37:20.524831 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:20.524796 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:37:21.534355 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:21.534313 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" event={"ID":"9000b594-3b6e-4044-b0e1-4532369efb03","Type":"ContainerStarted","Data":"a2d876765c4547708dc4399ec27c5d2ecab8f4d86daa42e879b33a41e84a23fc"} Jul 06 20:37:22.181253 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:22.181216 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:22.182432 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:22.182406 2583 patch_prober.go:28] interesting pod/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 06 20:37:22.182492 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:22.182457 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" podUID="9000b594-3b6e-4044-b0e1-4532369efb03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 06 20:37:22.446034 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:22.445940 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:23.181229 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:23.181198 2583 patch_prober.go:28] interesting pod/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 06 20:37:23.181603 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:23.181262 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" podUID="9000b594-3b6e-4044-b0e1-4532369efb03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 06 20:37:23.243430 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:23.243391 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:37:24.181676 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:24.181637 2583 patch_prober.go:28] interesting pod/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 06 20:37:24.182140 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:24.181694 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" podUID="9000b594-3b6e-4044-b0e1-4532369efb03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 06 20:37:25.181183 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:25.181153 2583 patch_prober.go:28] interesting pod/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 06 20:37:25.181359 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:25.181202 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" podUID="9000b594-3b6e-4044-b0e1-4532369efb03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 06 20:37:26.185404 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:26.185373 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:26.186323 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:26.186299 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:37:35.900597 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:35.900559 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:37:47.692303 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:37:47.692269 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:38:03.019620 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:38:03.019589 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:38:15.403662 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:38:15.403626 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:38:26.624812 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:38:26.624774 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:39:03.800438 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:03.800390 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql"] Jul 06 20:39:03.801056 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:03.800650 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" podUID="9000b594-3b6e-4044-b0e1-4532369efb03" containerName="istio-proxy" containerID="cri-o://a2d876765c4547708dc4399ec27c5d2ecab8f4d86daa42e879b33a41e84a23fc" gracePeriod=30 Jul 06 20:39:07.693353 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:07.693324 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:39:09.065520 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.065459 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:39:09.147985 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.147957 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9000b594-3b6e-4044-b0e1-4532369efb03-istio-podinfo\") pod \"9000b594-3b6e-4044-b0e1-4532369efb03\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " Jul 06 20:39:09.148115 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148001 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-credential-socket\") pod \"9000b594-3b6e-4044-b0e1-4532369efb03\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " Jul 06 20:39:09.148115 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148024 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9000b594-3b6e-4044-b0e1-4532369efb03-istiod-ca-cert\") pod \"9000b594-3b6e-4044-b0e1-4532369efb03\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " Jul 06 20:39:09.148115 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148039 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-workload-socket\") pod \"9000b594-3b6e-4044-b0e1-4532369efb03\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " Jul 06 20:39:09.148115 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148063 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-istio-data\") pod \"9000b594-3b6e-4044-b0e1-4532369efb03\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " Jul 06 20:39:09.148115 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148098 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7hs2\" (UniqueName: \"kubernetes.io/projected/9000b594-3b6e-4044-b0e1-4532369efb03-kube-api-access-x7hs2\") pod \"9000b594-3b6e-4044-b0e1-4532369efb03\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " Jul 06 20:39:09.148378 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148119 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-istio-envoy\") pod \"9000b594-3b6e-4044-b0e1-4532369efb03\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " Jul 06 20:39:09.148378 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148145 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-workload-certs\") pod \"9000b594-3b6e-4044-b0e1-4532369efb03\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " Jul 06 20:39:09.148378 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148231 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9000b594-3b6e-4044-b0e1-4532369efb03" (UID: "9000b594-3b6e-4044-b0e1-4532369efb03"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:39:09.148378 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148273 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9000b594-3b6e-4044-b0e1-4532369efb03-istio-token\") pod \"9000b594-3b6e-4044-b0e1-4532369efb03\" (UID: \"9000b594-3b6e-4044-b0e1-4532369efb03\") " Jul 06 20:39:09.148589 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148379 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9000b594-3b6e-4044-b0e1-4532369efb03-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9000b594-3b6e-4044-b0e1-4532369efb03" (UID: "9000b594-3b6e-4044-b0e1-4532369efb03"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:39:09.148589 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148474 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9000b594-3b6e-4044-b0e1-4532369efb03" (UID: "9000b594-3b6e-4044-b0e1-4532369efb03"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:39:09.148701 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148610 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:09.148701 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148628 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9000b594-3b6e-4044-b0e1-4532369efb03-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:09.148701 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148643 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:09.148701 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148661 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9000b594-3b6e-4044-b0e1-4532369efb03" (UID: "9000b594-3b6e-4044-b0e1-4532369efb03"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:39:09.149024 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.148998 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-istio-data" (OuterVolumeSpecName: "istio-data") pod "9000b594-3b6e-4044-b0e1-4532369efb03" (UID: "9000b594-3b6e-4044-b0e1-4532369efb03"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:39:09.150205 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.150181 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9000b594-3b6e-4044-b0e1-4532369efb03-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9000b594-3b6e-4044-b0e1-4532369efb03" (UID: "9000b594-3b6e-4044-b0e1-4532369efb03"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:39:09.150287 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.150274 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9000b594-3b6e-4044-b0e1-4532369efb03-kube-api-access-x7hs2" (OuterVolumeSpecName: "kube-api-access-x7hs2") pod "9000b594-3b6e-4044-b0e1-4532369efb03" (UID: "9000b594-3b6e-4044-b0e1-4532369efb03"). InnerVolumeSpecName "kube-api-access-x7hs2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:39:09.150799 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.150769 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9000b594-3b6e-4044-b0e1-4532369efb03" (UID: "9000b594-3b6e-4044-b0e1-4532369efb03"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:39:09.150901 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.150779 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9000b594-3b6e-4044-b0e1-4532369efb03-istio-token" (OuterVolumeSpecName: "istio-token") pod "9000b594-3b6e-4044-b0e1-4532369efb03" (UID: "9000b594-3b6e-4044-b0e1-4532369efb03"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:39:09.178288 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.178262 2583 generic.go:356] "Generic (PLEG): container finished" podID="9000b594-3b6e-4044-b0e1-4532369efb03" containerID="a2d876765c4547708dc4399ec27c5d2ecab8f4d86daa42e879b33a41e84a23fc" exitCode=0 Jul 06 20:39:09.178368 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.178305 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" Jul 06 20:39:09.178368 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.178350 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" event={"ID":"9000b594-3b6e-4044-b0e1-4532369efb03","Type":"ContainerDied","Data":"a2d876765c4547708dc4399ec27c5d2ecab8f4d86daa42e879b33a41e84a23fc"} Jul 06 20:39:09.178444 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.178397 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql" event={"ID":"9000b594-3b6e-4044-b0e1-4532369efb03","Type":"ContainerDied","Data":"a615e03655ea76d59953aa3bcbc13554f085568bf3743c913ada0ddbd71a7914"} Jul 06 20:39:09.178444 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.178421 2583 scope.go:117] "RemoveContainer" containerID="a2d876765c4547708dc4399ec27c5d2ecab8f4d86daa42e879b33a41e84a23fc" Jul 06 20:39:09.187676 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.187661 2583 scope.go:117] "RemoveContainer" containerID="ca0d7c87ff0c6bb2eff40a48c17c945a31a36f5350fa234ec8676cec9204417c" Jul 06 20:39:09.196393 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.196376 2583 scope.go:117] "RemoveContainer" containerID="a2d876765c4547708dc4399ec27c5d2ecab8f4d86daa42e879b33a41e84a23fc" Jul 06 20:39:09.196648 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:39:09.196631 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2d876765c4547708dc4399ec27c5d2ecab8f4d86daa42e879b33a41e84a23fc\": container with ID starting with a2d876765c4547708dc4399ec27c5d2ecab8f4d86daa42e879b33a41e84a23fc not found: ID does not exist" containerID="a2d876765c4547708dc4399ec27c5d2ecab8f4d86daa42e879b33a41e84a23fc" Jul 06 20:39:09.196693 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.196658 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d876765c4547708dc4399ec27c5d2ecab8f4d86daa42e879b33a41e84a23fc"} err="failed to get container status \"a2d876765c4547708dc4399ec27c5d2ecab8f4d86daa42e879b33a41e84a23fc\": rpc error: code = NotFound desc = could not find container \"a2d876765c4547708dc4399ec27c5d2ecab8f4d86daa42e879b33a41e84a23fc\": container with ID starting with a2d876765c4547708dc4399ec27c5d2ecab8f4d86daa42e879b33a41e84a23fc not found: ID does not exist" Jul 06 20:39:09.196693 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.196675 2583 scope.go:117] "RemoveContainer" containerID="ca0d7c87ff0c6bb2eff40a48c17c945a31a36f5350fa234ec8676cec9204417c" Jul 06 20:39:09.196931 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:39:09.196914 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca0d7c87ff0c6bb2eff40a48c17c945a31a36f5350fa234ec8676cec9204417c\": container with ID starting with ca0d7c87ff0c6bb2eff40a48c17c945a31a36f5350fa234ec8676cec9204417c not found: ID does not exist" containerID="ca0d7c87ff0c6bb2eff40a48c17c945a31a36f5350fa234ec8676cec9204417c" Jul 06 20:39:09.196991 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.196936 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca0d7c87ff0c6bb2eff40a48c17c945a31a36f5350fa234ec8676cec9204417c"} err="failed to get container status \"ca0d7c87ff0c6bb2eff40a48c17c945a31a36f5350fa234ec8676cec9204417c\": rpc error: code = NotFound desc = could not find container \"ca0d7c87ff0c6bb2eff40a48c17c945a31a36f5350fa234ec8676cec9204417c\": container with ID starting with ca0d7c87ff0c6bb2eff40a48c17c945a31a36f5350fa234ec8676cec9204417c not found: ID does not exist" Jul 06 20:39:09.202066 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.202045 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql"] Jul 06 20:39:09.217618 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.217599 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-cb2bb9-openshift-default-69b94d869-6xxql"] Jul 06 20:39:09.249655 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.249635 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9000b594-3b6e-4044-b0e1-4532369efb03-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:09.249655 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.249654 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9000b594-3b6e-4044-b0e1-4532369efb03-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:09.249779 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.249663 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:09.249779 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.249674 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7hs2\" (UniqueName: \"kubernetes.io/projected/9000b594-3b6e-4044-b0e1-4532369efb03-kube-api-access-x7hs2\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:09.249779 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.249682 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:09.249779 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.249690 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9000b594-3b6e-4044-b0e1-4532369efb03-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:09.484950 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:09.484924 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9000b594-3b6e-4044-b0e1-4532369efb03" path="/var/lib/kubelet/pods/9000b594-3b6e-4044-b0e1-4532369efb03/volumes" Jul 06 20:39:11.984610 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:11.984577 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf"] Jul 06 20:39:11.985227 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:11.985201 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ff9cda6-a8cc-4480-8314-f35eafa90ed3" containerName="istio-proxy" Jul 06 20:39:11.985227 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:11.985226 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ff9cda6-a8cc-4480-8314-f35eafa90ed3" containerName="istio-proxy" Jul 06 20:39:11.985388 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:11.985252 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9000b594-3b6e-4044-b0e1-4532369efb03" containerName="istio-proxy" Jul 06 20:39:11.985388 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:11.985260 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9000b594-3b6e-4044-b0e1-4532369efb03" containerName="istio-proxy" Jul 06 20:39:11.985388 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:11.985271 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9000b594-3b6e-4044-b0e1-4532369efb03" containerName="istio-proxy" Jul 06 20:39:11.985388 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:11.985277 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9000b594-3b6e-4044-b0e1-4532369efb03" containerName="istio-proxy" Jul 06 20:39:11.985388 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:11.985361 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9000b594-3b6e-4044-b0e1-4532369efb03" containerName="istio-proxy" Jul 06 20:39:11.985388 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:11.985369 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9000b594-3b6e-4044-b0e1-4532369efb03" containerName="istio-proxy" Jul 06 20:39:11.989832 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:11.989814 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:11.992324 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:11.992305 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-3d7651-openshift-default-dockercfg-zdd6g\"" Jul 06 20:39:12.000513 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.000493 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf"] Jul 06 20:39:12.073723 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.073686 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istiod-ca-cert\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.073905 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.073731 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-data\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.073905 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.073782 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-workload-socket\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.073905 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.073806 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-credential-socket\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.074074 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.073904 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-podinfo\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.074074 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.073954 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-envoy\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.074074 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.074001 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-token\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.074074 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.074034 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-workload-certs\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.074074 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.074064 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m42jw\" (UniqueName: \"kubernetes.io/projected/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-kube-api-access-m42jw\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.174605 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.174572 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m42jw\" (UniqueName: \"kubernetes.io/projected/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-kube-api-access-m42jw\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.174781 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.174632 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istiod-ca-cert\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.174781 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.174658 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-data\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.174781 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.174689 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-workload-socket\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.174781 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.174713 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-credential-socket\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.174781 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.174730 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-podinfo\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.174781 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.174756 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-envoy\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.175160 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.174800 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-token\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.175160 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.174934 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-workload-certs\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.175297 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.175276 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-data\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.175367 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.175345 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-workload-socket\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.175433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.175412 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-workload-certs\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.175481 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.175413 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-credential-socket\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.175481 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.175451 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istiod-ca-cert\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.177043 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.177018 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-envoy\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.177263 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.177244 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-podinfo\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.193045 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.193020 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-token\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.194098 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.194072 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m42jw\" (UniqueName: \"kubernetes.io/projected/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-kube-api-access-m42jw\") pod \"e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.300355 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.300291 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:12.450209 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.450179 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf"] Jul 06 20:39:12.451086 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:39:12.451060 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacdb30f2_d06c_49f1_9e0a_0494c40ce4a4.slice/crio-544ff09f5cea650110bce29c9b6f277c17b648507910cc691f1f0c2a7cc99b65 WatchSource:0}: Error finding container 544ff09f5cea650110bce29c9b6f277c17b648507910cc691f1f0c2a7cc99b65: Status 404 returned error can't find the container with id 544ff09f5cea650110bce29c9b6f277c17b648507910cc691f1f0c2a7cc99b65 Jul 06 20:39:12.453123 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.453086 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:39:12.453212 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.453159 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:39:12.453212 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:12.453190 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:39:13.204213 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:13.204181 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" event={"ID":"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4","Type":"ContainerStarted","Data":"18ae1a1ac5db3b22b68da72f499c4a6799fa4e2a101bcff6d8214583489bc360"} Jul 06 20:39:13.204632 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:13.204231 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" event={"ID":"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4","Type":"ContainerStarted","Data":"544ff09f5cea650110bce29c9b6f277c17b648507910cc691f1f0c2a7cc99b65"} Jul 06 20:39:13.230549 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:13.230501 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" podStartSLOduration=2.230486367 podStartE2EDuration="2.230486367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:39:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:39:13.227710461 +0000 UTC m=+2006.354622565" watchObservedRunningTime="2026-07-06 20:39:13.230486367 +0000 UTC m=+2006.357398470" Jul 06 20:39:13.300922 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:13.300900 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:13.305495 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:13.305465 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:14.210622 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:14.210592 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:14.211453 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:14.211435 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:20.312800 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:20.312766 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:39:28.303170 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:28.303098 2583 generic.go:356] "Generic (PLEG): container finished" podID="acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" containerID="18ae1a1ac5db3b22b68da72f499c4a6799fa4e2a101bcff6d8214583489bc360" exitCode=0 Jul 06 20:39:28.303578 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:28.303177 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" event={"ID":"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4","Type":"ContainerDied","Data":"18ae1a1ac5db3b22b68da72f499c4a6799fa4e2a101bcff6d8214583489bc360"} Jul 06 20:39:28.303625 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:28.303577 2583 scope.go:117] "RemoveContainer" containerID="18ae1a1ac5db3b22b68da72f499c4a6799fa4e2a101bcff6d8214583489bc360" Jul 06 20:39:28.304561 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:28.304534 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:39:28.304615 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:28.304600 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:39:28.304653 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:28.304639 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:39:29.210569 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:29.210533 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:29.310901 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:29.310851 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" event={"ID":"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4","Type":"ContainerStarted","Data":"6bba87825ee6d0990de5ef34f2903a0ff64b1f02a2c668ad7f231fb8547126df"} Jul 06 20:39:30.300676 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:30.300640 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:30.301891 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:30.301855 2583 patch_prober.go:28] interesting pod/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 20:39:30.302027 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:30.301924 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" podUID="acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 20:39:31.300789 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:31.300749 2583 patch_prober.go:28] interesting pod/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 20:39:31.301268 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:31.300829 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" podUID="acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 20:39:32.301572 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:32.301533 2583 patch_prober.go:28] interesting pod/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 20:39:32.301995 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:32.301602 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" podUID="acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 20:39:33.304424 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:33.304393 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:33.304854 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:33.304742 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:33.305469 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:33.305451 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:35.936764 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:35.936729 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:39:43.024472 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:43.024437 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf"] Jul 06 20:39:43.024908 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:43.024642 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" podUID="acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" containerName="istio-proxy" containerID="cri-o://6bba87825ee6d0990de5ef34f2903a0ff64b1f02a2c668ad7f231fb8547126df" gracePeriod=30 Jul 06 20:39:45.026751 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:45.026720 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:39:48.285261 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.285208 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:48.404834 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.404806 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-credential-socket\") pod \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " Jul 06 20:39:48.405010 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.404844 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-workload-socket\") pod \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " Jul 06 20:39:48.405010 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.404890 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m42jw\" (UniqueName: \"kubernetes.io/projected/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-kube-api-access-m42jw\") pod \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " Jul 06 20:39:48.405010 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.404951 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-envoy\") pod \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " Jul 06 20:39:48.405010 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.404988 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-data\") pod \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " Jul 06 20:39:48.405235 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.405025 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-podinfo\") pod \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " Jul 06 20:39:48.405235 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.405067 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istiod-ca-cert\") pod \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " Jul 06 20:39:48.405235 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.405094 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" (UID: "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:39:48.405235 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.405128 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" (UID: "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:39:48.405235 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.405178 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-token\") pod \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " Jul 06 20:39:48.405235 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.405204 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-workload-certs\") pod \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\" (UID: \"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4\") " Jul 06 20:39:48.405533 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.405468 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" (UID: "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:39:48.405592 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.405541 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:48.405592 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.405558 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:48.405592 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.405572 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:48.405744 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.405723 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" (UID: "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:39:48.406061 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.406037 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-data" (OuterVolumeSpecName: "istio-data") pod "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" (UID: "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:39:48.407295 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.407265 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-kube-api-access-m42jw" (OuterVolumeSpecName: "kube-api-access-m42jw") pod "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" (UID: "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4"). InnerVolumeSpecName "kube-api-access-m42jw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:39:48.407295 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.407274 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-token" (OuterVolumeSpecName: "istio-token") pod "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" (UID: "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:39:48.407578 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.407556 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" (UID: "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:39:48.407640 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.407568 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" (UID: "acdb30f2-d06c-49f1-9e0a-0494c40ce4a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:39:48.435741 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.435712 2583 generic.go:356] "Generic (PLEG): container finished" podID="acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" containerID="6bba87825ee6d0990de5ef34f2903a0ff64b1f02a2c668ad7f231fb8547126df" exitCode=0 Jul 06 20:39:48.435839 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.435762 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" Jul 06 20:39:48.435839 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.435793 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" event={"ID":"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4","Type":"ContainerDied","Data":"6bba87825ee6d0990de5ef34f2903a0ff64b1f02a2c668ad7f231fb8547126df"} Jul 06 20:39:48.435932 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.435841 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf" event={"ID":"acdb30f2-d06c-49f1-9e0a-0494c40ce4a4","Type":"ContainerDied","Data":"544ff09f5cea650110bce29c9b6f277c17b648507910cc691f1f0c2a7cc99b65"} Jul 06 20:39:48.435932 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.435863 2583 scope.go:117] "RemoveContainer" containerID="6bba87825ee6d0990de5ef34f2903a0ff64b1f02a2c668ad7f231fb8547126df" Jul 06 20:39:48.446950 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.446930 2583 scope.go:117] "RemoveContainer" containerID="18ae1a1ac5db3b22b68da72f499c4a6799fa4e2a101bcff6d8214583489bc360" Jul 06 20:39:48.456544 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.456528 2583 scope.go:117] "RemoveContainer" containerID="6bba87825ee6d0990de5ef34f2903a0ff64b1f02a2c668ad7f231fb8547126df" Jul 06 20:39:48.456800 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:39:48.456782 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bba87825ee6d0990de5ef34f2903a0ff64b1f02a2c668ad7f231fb8547126df\": container with ID starting with 6bba87825ee6d0990de5ef34f2903a0ff64b1f02a2c668ad7f231fb8547126df not found: ID does not exist" containerID="6bba87825ee6d0990de5ef34f2903a0ff64b1f02a2c668ad7f231fb8547126df" Jul 06 20:39:48.456860 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.456807 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bba87825ee6d0990de5ef34f2903a0ff64b1f02a2c668ad7f231fb8547126df"} err="failed to get container status \"6bba87825ee6d0990de5ef34f2903a0ff64b1f02a2c668ad7f231fb8547126df\": rpc error: code = NotFound desc = could not find container \"6bba87825ee6d0990de5ef34f2903a0ff64b1f02a2c668ad7f231fb8547126df\": container with ID starting with 6bba87825ee6d0990de5ef34f2903a0ff64b1f02a2c668ad7f231fb8547126df not found: ID does not exist" Jul 06 20:39:48.456860 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.456823 2583 scope.go:117] "RemoveContainer" containerID="18ae1a1ac5db3b22b68da72f499c4a6799fa4e2a101bcff6d8214583489bc360" Jul 06 20:39:48.457095 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:39:48.457079 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18ae1a1ac5db3b22b68da72f499c4a6799fa4e2a101bcff6d8214583489bc360\": container with ID starting with 18ae1a1ac5db3b22b68da72f499c4a6799fa4e2a101bcff6d8214583489bc360 not found: ID does not exist" containerID="18ae1a1ac5db3b22b68da72f499c4a6799fa4e2a101bcff6d8214583489bc360" Jul 06 20:39:48.457142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.457098 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ae1a1ac5db3b22b68da72f499c4a6799fa4e2a101bcff6d8214583489bc360"} err="failed to get container status \"18ae1a1ac5db3b22b68da72f499c4a6799fa4e2a101bcff6d8214583489bc360\": rpc error: code = NotFound desc = could not find container \"18ae1a1ac5db3b22b68da72f499c4a6799fa4e2a101bcff6d8214583489bc360\": container with ID starting with 18ae1a1ac5db3b22b68da72f499c4a6799fa4e2a101bcff6d8214583489bc360 not found: ID does not exist" Jul 06 20:39:48.466603 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.466582 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf"] Jul 06 20:39:48.468963 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.468944 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-3d7651-openshift-default-6776f47ff5-mxqpf"] Jul 06 20:39:48.506521 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.506502 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:48.506521 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.506521 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:48.506647 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.506531 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:48.506647 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.506539 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m42jw\" (UniqueName: \"kubernetes.io/projected/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-kube-api-access-m42jw\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:48.506647 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.506548 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:48.506647 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:48.506556 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:39:49.485259 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:49.485224 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" path="/var/lib/kubelet/pods/acdb30f2-d06c-49f1-9e0a-0494c40ce4a4/volumes" Jul 06 20:39:51.549593 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.549565 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s"] Jul 06 20:39:51.550025 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.550011 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" containerName="istio-proxy" Jul 06 20:39:51.550071 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.550027 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" containerName="istio-proxy" Jul 06 20:39:51.550106 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.550097 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" containerName="istio-proxy" Jul 06 20:39:51.553166 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.553149 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.555525 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.555501 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-37e178-openshift-default-dockercfg-zsb9f\"" Jul 06 20:39:51.564411 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.564392 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s"] Jul 06 20:39:51.635438 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.635404 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-credential-socket\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.635592 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.635454 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-workload-certs\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.635592 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.635510 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-envoy\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.635592 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.635580 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-podinfo\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.635814 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.635616 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-workload-socket\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.635814 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.635652 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zl6t\" (UniqueName: \"kubernetes.io/projected/19716d30-2da7-4906-9bc9-942188dfa4e1-kube-api-access-6zl6t\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.635814 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.635703 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-token\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.635814 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.635729 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19716d30-2da7-4906-9bc9-942188dfa4e1-istiod-ca-cert\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.635814 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.635753 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-data\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.736883 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.736846 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-token\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.737009 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.736900 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19716d30-2da7-4906-9bc9-942188dfa4e1-istiod-ca-cert\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.737009 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.736928 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-data\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.737009 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.736979 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-credential-socket\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.737009 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.737007 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-workload-certs\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.737234 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.737053 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-envoy\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.737234 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.737111 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-podinfo\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.737234 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.737147 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-workload-socket\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.737234 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.737184 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zl6t\" (UniqueName: \"kubernetes.io/projected/19716d30-2da7-4906-9bc9-942188dfa4e1-kube-api-access-6zl6t\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.737441 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.737397 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-data\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.737498 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.737432 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-credential-socket\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.737651 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.737632 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-workload-certs\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.737767 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.737745 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-workload-socket\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.737826 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.737789 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19716d30-2da7-4906-9bc9-942188dfa4e1-istiod-ca-cert\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.739344 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.739317 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-envoy\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.739636 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.739619 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-podinfo\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.758053 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.758027 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zl6t\" (UniqueName: \"kubernetes.io/projected/19716d30-2da7-4906-9bc9-942188dfa4e1-kube-api-access-6zl6t\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.759112 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.759084 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-token\") pod \"e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:51.863970 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:51.863908 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:52.018034 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:52.018004 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s"] Jul 06 20:39:52.019269 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:39:52.019240 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19716d30_2da7_4906_9bc9_942188dfa4e1.slice/crio-72635c9ad0e453775af50483ced2a13b91c0690518f1d0eaf00ea5e5444cdaf0 WatchSource:0}: Error finding container 72635c9ad0e453775af50483ced2a13b91c0690518f1d0eaf00ea5e5444cdaf0: Status 404 returned error can't find the container with id 72635c9ad0e453775af50483ced2a13b91c0690518f1d0eaf00ea5e5444cdaf0 Jul 06 20:39:52.021399 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:52.021366 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:39:52.021504 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:52.021441 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:39:52.021504 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:52.021482 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:39:52.464543 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:52.464503 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" event={"ID":"19716d30-2da7-4906-9bc9-942188dfa4e1","Type":"ContainerStarted","Data":"91991ed1297c839d02c2128ff6ec1bd65697e31a49164216f36a05e939498fa3"} Jul 06 20:39:52.464709 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:52.464569 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" event={"ID":"19716d30-2da7-4906-9bc9-942188dfa4e1","Type":"ContainerStarted","Data":"72635c9ad0e453775af50483ced2a13b91c0690518f1d0eaf00ea5e5444cdaf0"} Jul 06 20:39:52.489137 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:52.489092 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" podStartSLOduration=1.48907657 podStartE2EDuration="1.48907657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:39:52.487593777 +0000 UTC m=+2045.614505881" watchObservedRunningTime="2026-07-06 20:39:52.48907657 +0000 UTC m=+2045.615988673" Jul 06 20:39:52.864466 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:52.864428 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:53.868868 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:53.868836 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:54.476521 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:54.476492 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:54.477378 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:54.477358 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:39:56.927042 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:39:56.926998 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:40:04.547092 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:04.547059 2583 generic.go:356] "Generic (PLEG): container finished" podID="19716d30-2da7-4906-9bc9-942188dfa4e1" containerID="91991ed1297c839d02c2128ff6ec1bd65697e31a49164216f36a05e939498fa3" exitCode=0 Jul 06 20:40:04.547463 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:04.547130 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" event={"ID":"19716d30-2da7-4906-9bc9-942188dfa4e1","Type":"ContainerDied","Data":"91991ed1297c839d02c2128ff6ec1bd65697e31a49164216f36a05e939498fa3"} Jul 06 20:40:04.547516 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:04.547490 2583 scope.go:117] "RemoveContainer" containerID="91991ed1297c839d02c2128ff6ec1bd65697e31a49164216f36a05e939498fa3" Jul 06 20:40:04.548510 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:04.548483 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:40:04.548560 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:04.548547 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:40:04.548608 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:04.548595 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 20:40:05.558405 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:05.558368 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" event={"ID":"19716d30-2da7-4906-9bc9-942188dfa4e1","Type":"ContainerStarted","Data":"6973ede72015356c27bdedf4fca4d65ab07bafe38038b697a1427b2dced4db3d"} Jul 06 20:40:05.864750 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:05.864660 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:40:05.866005 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:05.865979 2583 patch_prober.go:28] interesting pod/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 20:40:05.866135 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:05.866027 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" podUID="19716d30-2da7-4906-9bc9-942188dfa4e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 20:40:06.865247 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:06.865215 2583 patch_prober.go:28] interesting pod/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 20:40:06.865621 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:06.865277 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" podUID="19716d30-2da7-4906-9bc9-942188dfa4e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 20:40:07.865172 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:07.865132 2583 patch_prober.go:28] interesting pod/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 20:40:07.865339 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:07.865190 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" podUID="19716d30-2da7-4906-9bc9-942188dfa4e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 20:40:08.864597 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:08.864562 2583 patch_prober.go:28] interesting pod/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 20:40:08.864798 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:08.864621 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" podUID="19716d30-2da7-4906-9bc9-942188dfa4e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 20:40:09.477239 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:09.477205 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:40:09.868152 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:09.868083 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:40:09.868978 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:09.868961 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:40:14.300895 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:14.300205 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:40:33.826237 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:33.826206 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:40:47.521857 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:47.521834 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:40:47.530572 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:47.530548 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:40:50.495974 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:40:50.495938 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:41:05.298562 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:41:05.298530 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:41:17.936042 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:41:17.936011 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:41:33.293611 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:41:33.293582 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:41:49.794141 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:41:49.794066 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:42:05.900216 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:42:05.900182 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:42:23.096301 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:42:23.096264 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:42:34.820426 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:42:34.820392 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:42:50.705360 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:42:50.705326 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:42:58.425375 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:42:58.425335 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:43:30.410571 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:43:30.410540 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:43:49.864376 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:43:49.864334 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:44:05.408660 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:44:05.408624 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:44:19.633825 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:44:19.633789 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:44:33.935116 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:44:33.935071 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:44:45.724481 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:44:45.724386 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:45:00.150195 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.150158 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp"] Jul 06 20:45:00.150670 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.150653 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" containerName="istio-proxy" Jul 06 20:45:00.150725 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.150672 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" containerName="istio-proxy" Jul 06 20:45:00.150795 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.150785 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="acdb30f2-d06c-49f1-9e0a-0494c40ce4a4" containerName="istio-proxy" Jul 06 20:45:00.154079 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.154055 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp" Jul 06 20:45:00.155784 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.155760 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl"] Jul 06 20:45:00.158227 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.158197 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hsq27\"" Jul 06 20:45:00.158590 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.158572 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 06 20:45:00.160430 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.159704 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 06 20:45:00.165819 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.165793 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl" Jul 06 20:45:00.172255 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.172229 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp"] Jul 06 20:45:00.174987 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.174967 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl"] Jul 06 20:45:00.336234 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.336208 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd5gq\" (UniqueName: \"kubernetes.io/projected/04da2b6d-76e5-40d3-bd6e-cbb446a56eb6-kube-api-access-cd5gq\") pod \"maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp\" (UID: \"04da2b6d-76e5-40d3-bd6e-cbb446a56eb6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp" Jul 06 20:45:00.336391 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.336237 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp7r8\" (UniqueName: \"kubernetes.io/projected/8bf5f93b-f441-415d-bb46-b87d5f4ecc1c-kube-api-access-hp7r8\") pod \"maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl\" (UID: \"8bf5f93b-f441-415d-bb46-b87d5f4ecc1c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl" Jul 06 20:45:00.437423 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.437355 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd5gq\" (UniqueName: \"kubernetes.io/projected/04da2b6d-76e5-40d3-bd6e-cbb446a56eb6-kube-api-access-cd5gq\") pod \"maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp\" (UID: \"04da2b6d-76e5-40d3-bd6e-cbb446a56eb6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp" Jul 06 20:45:00.437423 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.437387 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp7r8\" (UniqueName: \"kubernetes.io/projected/8bf5f93b-f441-415d-bb46-b87d5f4ecc1c-kube-api-access-hp7r8\") pod \"maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl\" (UID: \"8bf5f93b-f441-415d-bb46-b87d5f4ecc1c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl" Jul 06 20:45:00.447961 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.447941 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp7r8\" (UniqueName: \"kubernetes.io/projected/8bf5f93b-f441-415d-bb46-b87d5f4ecc1c-kube-api-access-hp7r8\") pod \"maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl\" (UID: \"8bf5f93b-f441-415d-bb46-b87d5f4ecc1c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl" Jul 06 20:45:00.448075 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.448021 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd5gq\" (UniqueName: \"kubernetes.io/projected/04da2b6d-76e5-40d3-bd6e-cbb446a56eb6-kube-api-access-cd5gq\") pod \"maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp\" (UID: \"04da2b6d-76e5-40d3-bd6e-cbb446a56eb6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp" Jul 06 20:45:00.473195 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.473173 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp" Jul 06 20:45:00.483421 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.483403 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl" Jul 06 20:45:00.630584 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.630551 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp"] Jul 06 20:45:00.631535 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:45:00.631511 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04da2b6d_76e5_40d3_bd6e_cbb446a56eb6.slice/crio-2a57aa1023521a1132942e29fbfa4193de2cdd67d9ff314103f09e07422675dd WatchSource:0}: Error finding container 2a57aa1023521a1132942e29fbfa4193de2cdd67d9ff314103f09e07422675dd: Status 404 returned error can't find the container with id 2a57aa1023521a1132942e29fbfa4193de2cdd67d9ff314103f09e07422675dd Jul 06 20:45:00.653166 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:00.653146 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl"] Jul 06 20:45:00.656001 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:45:00.655976 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf5f93b_f441_415d_bb46_b87d5f4ecc1c.slice/crio-ab15c5515661ca69826ad14583d238505d0dbce812fb30f2f7c60c8dfe1261c7 WatchSource:0}: Error finding container ab15c5515661ca69826ad14583d238505d0dbce812fb30f2f7c60c8dfe1261c7: Status 404 returned error can't find the container with id ab15c5515661ca69826ad14583d238505d0dbce812fb30f2f7c60c8dfe1261c7 Jul 06 20:45:01.322931 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:01.322902 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp" event={"ID":"04da2b6d-76e5-40d3-bd6e-cbb446a56eb6","Type":"ContainerStarted","Data":"5a0d669669bfb2c309400bad3c08eb23430dafab05978c44ad9c3e04541f1ee9"} Jul 06 20:45:01.323329 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:01.322957 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp" event={"ID":"04da2b6d-76e5-40d3-bd6e-cbb446a56eb6","Type":"ContainerStarted","Data":"2a57aa1023521a1132942e29fbfa4193de2cdd67d9ff314103f09e07422675dd"} Jul 06 20:45:01.324508 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:01.324488 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl" event={"ID":"8bf5f93b-f441-415d-bb46-b87d5f4ecc1c","Type":"ContainerStarted","Data":"e74bbf91f3378551e3ff1e83a8897b4c9059d1f55494074f72c9f4e3afaf5ec8"} Jul 06 20:45:01.324580 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:01.324525 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl" event={"ID":"8bf5f93b-f441-415d-bb46-b87d5f4ecc1c","Type":"ContainerStarted","Data":"ab15c5515661ca69826ad14583d238505d0dbce812fb30f2f7c60c8dfe1261c7"} Jul 06 20:45:01.340764 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:01.340723 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp" podStartSLOduration=1.340710278 podStartE2EDuration="1.340710278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:45:01.340363383 +0000 UTC m=+2354.467275484" watchObservedRunningTime="2026-07-06 20:45:01.340710278 +0000 UTC m=+2354.467622381" Jul 06 20:45:01.357744 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:01.357702 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl" podStartSLOduration=1.357690083 podStartE2EDuration="1.357690083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:45:01.356232691 +0000 UTC m=+2354.483144796" watchObservedRunningTime="2026-07-06 20:45:01.357690083 +0000 UTC m=+2354.484602219" Jul 06 20:45:20.300435 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:20.300389 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:45:38.793656 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:38.793623 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:45:47.567665 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:47.567636 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:45:47.576071 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:47.576049 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:45:52.691320 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:45:52.691283 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:46:00.015754 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:00.015722 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp"] Jul 06 20:46:00.016253 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:00.015973 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp" podUID="04da2b6d-76e5-40d3-bd6e-cbb446a56eb6" containerName="cleanup" containerID="cri-o://5a0d669669bfb2c309400bad3c08eb23430dafab05978c44ad9c3e04541f1ee9" gracePeriod=30 Jul 06 20:46:00.018297 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:00.018277 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl"] Jul 06 20:46:00.018451 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:00.018431 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl" podUID="8bf5f93b-f441-415d-bb46-b87d5f4ecc1c" containerName="cleanup" containerID="cri-o://e74bbf91f3378551e3ff1e83a8897b4c9059d1f55494074f72c9f4e3afaf5ec8" gracePeriod=30 Jul 06 20:46:05.192356 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:05.192323 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:46:30.210051 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.210029 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp" Jul 06 20:46:30.223367 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.223347 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl" Jul 06 20:46:30.276409 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.276349 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hp7r8\" (UniqueName: \"kubernetes.io/projected/8bf5f93b-f441-415d-bb46-b87d5f4ecc1c-kube-api-access-hp7r8\") pod \"8bf5f93b-f441-415d-bb46-b87d5f4ecc1c\" (UID: \"8bf5f93b-f441-415d-bb46-b87d5f4ecc1c\") " Jul 06 20:46:30.276525 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.276437 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cd5gq\" (UniqueName: \"kubernetes.io/projected/04da2b6d-76e5-40d3-bd6e-cbb446a56eb6-kube-api-access-cd5gq\") pod \"04da2b6d-76e5-40d3-bd6e-cbb446a56eb6\" (UID: \"04da2b6d-76e5-40d3-bd6e-cbb446a56eb6\") " Jul 06 20:46:30.278344 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.278321 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bf5f93b-f441-415d-bb46-b87d5f4ecc1c-kube-api-access-hp7r8" (OuterVolumeSpecName: "kube-api-access-hp7r8") pod "8bf5f93b-f441-415d-bb46-b87d5f4ecc1c" (UID: "8bf5f93b-f441-415d-bb46-b87d5f4ecc1c"). InnerVolumeSpecName "kube-api-access-hp7r8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:46:30.278427 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.278362 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04da2b6d-76e5-40d3-bd6e-cbb446a56eb6-kube-api-access-cd5gq" (OuterVolumeSpecName: "kube-api-access-cd5gq") pod "04da2b6d-76e5-40d3-bd6e-cbb446a56eb6" (UID: "04da2b6d-76e5-40d3-bd6e-cbb446a56eb6"). InnerVolumeSpecName "kube-api-access-cd5gq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:46:30.377991 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.377967 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp7r8\" (UniqueName: \"kubernetes.io/projected/8bf5f93b-f441-415d-bb46-b87d5f4ecc1c-kube-api-access-hp7r8\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:46:30.377991 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.377990 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cd5gq\" (UniqueName: \"kubernetes.io/projected/04da2b6d-76e5-40d3-bd6e-cbb446a56eb6-kube-api-access-cd5gq\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:46:30.861587 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.861553 2583 generic.go:356] "Generic (PLEG): container finished" podID="8bf5f93b-f441-415d-bb46-b87d5f4ecc1c" containerID="e74bbf91f3378551e3ff1e83a8897b4c9059d1f55494074f72c9f4e3afaf5ec8" exitCode=137 Jul 06 20:46:30.861766 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.861594 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl" Jul 06 20:46:30.861766 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.861635 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl" event={"ID":"8bf5f93b-f441-415d-bb46-b87d5f4ecc1c","Type":"ContainerDied","Data":"e74bbf91f3378551e3ff1e83a8897b4c9059d1f55494074f72c9f4e3afaf5ec8"} Jul 06 20:46:30.861766 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.861692 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl" event={"ID":"8bf5f93b-f441-415d-bb46-b87d5f4ecc1c","Type":"ContainerDied","Data":"ab15c5515661ca69826ad14583d238505d0dbce812fb30f2f7c60c8dfe1261c7"} Jul 06 20:46:30.861766 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.861716 2583 scope.go:117] "RemoveContainer" containerID="e74bbf91f3378551e3ff1e83a8897b4c9059d1f55494074f72c9f4e3afaf5ec8" Jul 06 20:46:30.863397 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.863371 2583 generic.go:356] "Generic (PLEG): container finished" podID="04da2b6d-76e5-40d3-bd6e-cbb446a56eb6" containerID="5a0d669669bfb2c309400bad3c08eb23430dafab05978c44ad9c3e04541f1ee9" exitCode=137 Jul 06 20:46:30.863536 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.863407 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp" Jul 06 20:46:30.863536 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.863459 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp" event={"ID":"04da2b6d-76e5-40d3-bd6e-cbb446a56eb6","Type":"ContainerDied","Data":"5a0d669669bfb2c309400bad3c08eb23430dafab05978c44ad9c3e04541f1ee9"} Jul 06 20:46:30.863536 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.863516 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp" event={"ID":"04da2b6d-76e5-40d3-bd6e-cbb446a56eb6","Type":"ContainerDied","Data":"2a57aa1023521a1132942e29fbfa4193de2cdd67d9ff314103f09e07422675dd"} Jul 06 20:46:30.874832 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.874816 2583 scope.go:117] "RemoveContainer" containerID="e74bbf91f3378551e3ff1e83a8897b4c9059d1f55494074f72c9f4e3afaf5ec8" Jul 06 20:46:30.875155 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:46:30.875113 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e74bbf91f3378551e3ff1e83a8897b4c9059d1f55494074f72c9f4e3afaf5ec8\": container with ID starting with e74bbf91f3378551e3ff1e83a8897b4c9059d1f55494074f72c9f4e3afaf5ec8 not found: ID does not exist" containerID="e74bbf91f3378551e3ff1e83a8897b4c9059d1f55494074f72c9f4e3afaf5ec8" Jul 06 20:46:30.875275 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.875155 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e74bbf91f3378551e3ff1e83a8897b4c9059d1f55494074f72c9f4e3afaf5ec8"} err="failed to get container status \"e74bbf91f3378551e3ff1e83a8897b4c9059d1f55494074f72c9f4e3afaf5ec8\": rpc error: code = NotFound desc = could not find container \"e74bbf91f3378551e3ff1e83a8897b4c9059d1f55494074f72c9f4e3afaf5ec8\": container with ID starting with e74bbf91f3378551e3ff1e83a8897b4c9059d1f55494074f72c9f4e3afaf5ec8 not found: ID does not exist" Jul 06 20:46:30.875275 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.875180 2583 scope.go:117] "RemoveContainer" containerID="5a0d669669bfb2c309400bad3c08eb23430dafab05978c44ad9c3e04541f1ee9" Jul 06 20:46:30.883433 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.883416 2583 scope.go:117] "RemoveContainer" containerID="5a0d669669bfb2c309400bad3c08eb23430dafab05978c44ad9c3e04541f1ee9" Jul 06 20:46:30.883664 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:46:30.883644 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a0d669669bfb2c309400bad3c08eb23430dafab05978c44ad9c3e04541f1ee9\": container with ID starting with 5a0d669669bfb2c309400bad3c08eb23430dafab05978c44ad9c3e04541f1ee9 not found: ID does not exist" containerID="5a0d669669bfb2c309400bad3c08eb23430dafab05978c44ad9c3e04541f1ee9" Jul 06 20:46:30.883706 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.883673 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a0d669669bfb2c309400bad3c08eb23430dafab05978c44ad9c3e04541f1ee9"} err="failed to get container status \"5a0d669669bfb2c309400bad3c08eb23430dafab05978c44ad9c3e04541f1ee9\": rpc error: code = NotFound desc = could not find container \"5a0d669669bfb2c309400bad3c08eb23430dafab05978c44ad9c3e04541f1ee9\": container with ID starting with 5a0d669669bfb2c309400bad3c08eb23430dafab05978c44ad9c3e04541f1ee9 not found: ID does not exist" Jul 06 20:46:30.893596 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.893576 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl"] Jul 06 20:46:30.904329 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.899563 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-67755c-29722845-289vl"] Jul 06 20:46:30.910858 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.910839 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp"] Jul 06 20:46:30.924582 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:30.924560 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e178-29722845-km2fp"] Jul 06 20:46:31.486447 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:31.486410 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04da2b6d-76e5-40d3-bd6e-cbb446a56eb6" path="/var/lib/kubelet/pods/04da2b6d-76e5-40d3-bd6e-cbb446a56eb6/volumes" Jul 06 20:46:31.486844 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:31.486827 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bf5f93b-f441-415d-bb46-b87d5f4ecc1c" path="/var/lib/kubelet/pods/8bf5f93b-f441-415d-bb46-b87d5f4ecc1c/volumes" Jul 06 20:46:43.897485 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:43.897452 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s"] Jul 06 20:46:43.898037 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:43.897661 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" podUID="19716d30-2da7-4906-9bc9-942188dfa4e1" containerName="istio-proxy" containerID="cri-o://6973ede72015356c27bdedf4fca4d65ab07bafe38038b697a1427b2dced4db3d" gracePeriod=30 Jul 06 20:46:47.898885 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:47.898853 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cdjxm"] Jul 06 20:46:49.161477 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.161456 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:46:49.237682 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.237654 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zl6t\" (UniqueName: \"kubernetes.io/projected/19716d30-2da7-4906-9bc9-942188dfa4e1-kube-api-access-6zl6t\") pod \"19716d30-2da7-4906-9bc9-942188dfa4e1\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " Jul 06 20:46:49.237839 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.237706 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-token\") pod \"19716d30-2da7-4906-9bc9-942188dfa4e1\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " Jul 06 20:46:49.237839 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.237740 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-data\") pod \"19716d30-2da7-4906-9bc9-942188dfa4e1\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " Jul 06 20:46:49.237839 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.237757 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-credential-socket\") pod \"19716d30-2da7-4906-9bc9-942188dfa4e1\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " Jul 06 20:46:49.237839 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.237785 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-envoy\") pod \"19716d30-2da7-4906-9bc9-942188dfa4e1\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " Jul 06 20:46:49.237839 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.237816 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-workload-socket\") pod \"19716d30-2da7-4906-9bc9-942188dfa4e1\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " Jul 06 20:46:49.238142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.237840 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-podinfo\") pod \"19716d30-2da7-4906-9bc9-942188dfa4e1\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " Jul 06 20:46:49.238142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.237868 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19716d30-2da7-4906-9bc9-942188dfa4e1-istiod-ca-cert\") pod \"19716d30-2da7-4906-9bc9-942188dfa4e1\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " Jul 06 20:46:49.238142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.237948 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-workload-certs\") pod \"19716d30-2da7-4906-9bc9-942188dfa4e1\" (UID: \"19716d30-2da7-4906-9bc9-942188dfa4e1\") " Jul 06 20:46:49.238142 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.238077 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "19716d30-2da7-4906-9bc9-942188dfa4e1" (UID: "19716d30-2da7-4906-9bc9-942188dfa4e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:46:49.238359 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.238292 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:46:49.238688 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.238627 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "19716d30-2da7-4906-9bc9-942188dfa4e1" (UID: "19716d30-2da7-4906-9bc9-942188dfa4e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:46:49.238688 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.238660 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19716d30-2da7-4906-9bc9-942188dfa4e1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "19716d30-2da7-4906-9bc9-942188dfa4e1" (UID: "19716d30-2da7-4906-9bc9-942188dfa4e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:46:49.238847 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.238761 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "19716d30-2da7-4906-9bc9-942188dfa4e1" (UID: "19716d30-2da7-4906-9bc9-942188dfa4e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:46:49.238922 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.238847 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-data" (OuterVolumeSpecName: "istio-data") pod "19716d30-2da7-4906-9bc9-942188dfa4e1" (UID: "19716d30-2da7-4906-9bc9-942188dfa4e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:46:49.239998 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.239977 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "19716d30-2da7-4906-9bc9-942188dfa4e1" (UID: "19716d30-2da7-4906-9bc9-942188dfa4e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:46:49.240094 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.240069 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19716d30-2da7-4906-9bc9-942188dfa4e1-kube-api-access-6zl6t" (OuterVolumeSpecName: "kube-api-access-6zl6t") pod "19716d30-2da7-4906-9bc9-942188dfa4e1" (UID: "19716d30-2da7-4906-9bc9-942188dfa4e1"). InnerVolumeSpecName "kube-api-access-6zl6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:46:49.240341 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.240320 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "19716d30-2da7-4906-9bc9-942188dfa4e1" (UID: "19716d30-2da7-4906-9bc9-942188dfa4e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:46:49.240392 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.240349 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "19716d30-2da7-4906-9bc9-942188dfa4e1" (UID: "19716d30-2da7-4906-9bc9-942188dfa4e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:46:49.339023 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.338957 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:46:49.339023 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.338980 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:46:49.339023 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.338990 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19716d30-2da7-4906-9bc9-942188dfa4e1-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:46:49.339023 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.338998 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:46:49.339023 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.339006 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zl6t\" (UniqueName: \"kubernetes.io/projected/19716d30-2da7-4906-9bc9-942188dfa4e1-kube-api-access-6zl6t\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:46:49.339023 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.339015 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:46:49.339023 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.339023 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:46:49.339023 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.339031 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19716d30-2da7-4906-9bc9-942188dfa4e1-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\"" Jul 06 20:46:49.989297 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.989265 2583 generic.go:356] "Generic (PLEG): container finished" podID="19716d30-2da7-4906-9bc9-942188dfa4e1" containerID="6973ede72015356c27bdedf4fca4d65ab07bafe38038b697a1427b2dced4db3d" exitCode=0 Jul 06 20:46:49.989491 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.989312 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" Jul 06 20:46:49.989491 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.989353 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" event={"ID":"19716d30-2da7-4906-9bc9-942188dfa4e1","Type":"ContainerDied","Data":"6973ede72015356c27bdedf4fca4d65ab07bafe38038b697a1427b2dced4db3d"} Jul 06 20:46:49.989491 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.989416 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s" event={"ID":"19716d30-2da7-4906-9bc9-942188dfa4e1","Type":"ContainerDied","Data":"72635c9ad0e453775af50483ced2a13b91c0690518f1d0eaf00ea5e5444cdaf0"} Jul 06 20:46:49.989491 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.989448 2583 scope.go:117] "RemoveContainer" containerID="6973ede72015356c27bdedf4fca4d65ab07bafe38038b697a1427b2dced4db3d" Jul 06 20:46:49.999193 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:49.999176 2583 scope.go:117] "RemoveContainer" containerID="91991ed1297c839d02c2128ff6ec1bd65697e31a49164216f36a05e939498fa3" Jul 06 20:46:50.007323 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:50.007307 2583 scope.go:117] "RemoveContainer" containerID="6973ede72015356c27bdedf4fca4d65ab07bafe38038b697a1427b2dced4db3d" Jul 06 20:46:50.007574 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:46:50.007552 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6973ede72015356c27bdedf4fca4d65ab07bafe38038b697a1427b2dced4db3d\": container with ID starting with 6973ede72015356c27bdedf4fca4d65ab07bafe38038b697a1427b2dced4db3d not found: ID does not exist" containerID="6973ede72015356c27bdedf4fca4d65ab07bafe38038b697a1427b2dced4db3d" Jul 06 20:46:50.007636 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:50.007583 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6973ede72015356c27bdedf4fca4d65ab07bafe38038b697a1427b2dced4db3d"} err="failed to get container status \"6973ede72015356c27bdedf4fca4d65ab07bafe38038b697a1427b2dced4db3d\": rpc error: code = NotFound desc = could not find container \"6973ede72015356c27bdedf4fca4d65ab07bafe38038b697a1427b2dced4db3d\": container with ID starting with 6973ede72015356c27bdedf4fca4d65ab07bafe38038b697a1427b2dced4db3d not found: ID does not exist" Jul 06 20:46:50.007636 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:50.007597 2583 scope.go:117] "RemoveContainer" containerID="91991ed1297c839d02c2128ff6ec1bd65697e31a49164216f36a05e939498fa3" Jul 06 20:46:50.007813 ip-10-0-130-133 kubenswrapper[2583]: E0706 20:46:50.007795 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91991ed1297c839d02c2128ff6ec1bd65697e31a49164216f36a05e939498fa3\": container with ID starting with 91991ed1297c839d02c2128ff6ec1bd65697e31a49164216f36a05e939498fa3 not found: ID does not exist" containerID="91991ed1297c839d02c2128ff6ec1bd65697e31a49164216f36a05e939498fa3" Jul 06 20:46:50.007860 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:50.007820 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91991ed1297c839d02c2128ff6ec1bd65697e31a49164216f36a05e939498fa3"} err="failed to get container status \"91991ed1297c839d02c2128ff6ec1bd65697e31a49164216f36a05e939498fa3\": rpc error: code = NotFound desc = could not find container \"91991ed1297c839d02c2128ff6ec1bd65697e31a49164216f36a05e939498fa3\": container with ID starting with 91991ed1297c839d02c2128ff6ec1bd65697e31a49164216f36a05e939498fa3 not found: ID does not exist" Jul 06 20:46:50.015449 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:50.015429 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s"] Jul 06 20:46:50.022318 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:50.022298 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-37e178-openshift-default-746d4d5df5-v2k8s"] Jul 06 20:46:51.485260 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:51.485228 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19716d30-2da7-4906-9bc9-942188dfa4e1" path="/var/lib/kubelet/pods/19716d30-2da7-4906-9bc9-942188dfa4e1/volumes" Jul 06 20:46:52.546284 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:52.546252 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5d5958db9b-tfpxl_b271f5aa-0263-47ea-9f03-b2dcbc80a78e/authorino/0.log" Jul 06 20:46:57.218868 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:57.218841 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bfvds_73340a04-d613-40d5-85ab-7c54daeaa757/manager/1.log" Jul 06 20:46:57.336082 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:57.336051 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7vpbr_4c49034e-79c9-407b-b068-96bbe8d61dac/manager/1.log" Jul 06 20:46:57.701516 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:57.701490 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-np686_ed7b2a0f-049e-4874-8a09-076638035822/manager/1.log" Jul 06 20:46:58.064185 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:58.064160 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cbf59454-jvfwm_a5f8a799-9d7a-498c-8e6d-8baa3036efab/copy-manifests/0.log" Jul 06 20:46:58.071973 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:58.071952 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cbf59454-jvfwm_a5f8a799-9d7a-498c-8e6d-8baa3036efab/manager/1.log" Jul 06 20:46:58.849631 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:58.849599 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd_5884ee3f-4633-4425-a422-abcb819e01b5/util/0.log" Jul 06 20:46:58.857291 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:58.857268 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd_5884ee3f-4633-4425-a422-abcb819e01b5/pull/0.log" Jul 06 20:46:58.865023 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:58.865004 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd_5884ee3f-4633-4425-a422-abcb819e01b5/extract/0.log" Jul 06 20:46:58.983818 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:58.983789 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6_6603e100-ee5e-4c3d-9063-0b638e10b392/util/0.log" Jul 06 20:46:58.991727 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:58.991705 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6_6603e100-ee5e-4c3d-9063-0b638e10b392/pull/0.log" Jul 06 20:46:58.999494 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:58.999475 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6_6603e100-ee5e-4c3d-9063-0b638e10b392/extract/0.log" Jul 06 20:46:59.106389 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:59.106338 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96_6485c000-68a5-4ee0-a899-9925bee9b133/util/0.log" Jul 06 20:46:59.114056 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:59.114034 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96_6485c000-68a5-4ee0-a899-9925bee9b133/pull/0.log" Jul 06 20:46:59.121173 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:59.121158 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96_6485c000-68a5-4ee0-a899-9925bee9b133/extract/0.log" Jul 06 20:46:59.230036 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:59.230016 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5_602cda8d-f8c4-4a6a-9a04-5759fc1af23c/util/0.log" Jul 06 20:46:59.238065 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:59.238050 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5_602cda8d-f8c4-4a6a-9a04-5759fc1af23c/pull/0.log" Jul 06 20:46:59.245471 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:59.245456 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5_602cda8d-f8c4-4a6a-9a04-5759fc1af23c/extract/0.log" Jul 06 20:46:59.361936 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:59.361864 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5d5958db9b-tfpxl_b271f5aa-0263-47ea-9f03-b2dcbc80a78e/authorino/0.log" Jul 06 20:46:59.478464 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:59.478445 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tj6k6_ba75fdd1-3a5a-46cb-a655-a46013cf495e/manager/1.log" Jul 06 20:46:59.588172 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:59.588153 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-drtjk_0732e11e-ca32-46dc-b9fe-a7ad6ef72eae/manager/1.log" Jul 06 20:46:59.703832 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:59.703769 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-b5j27_7e2f1dd2-f940-4c42-89f3-e8691ec33679/kuadrant-console-plugin/0.log" Jul 06 20:46:59.831305 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:59.831283 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4khpv_d4d809a6-2593-4470-b8a9-49c663b07b0e/registry-server/0.log" Jul 06 20:46:59.965330 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:46:59.965277 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-pzfd9_a9399ca9-ef00-49bc-97f9-ce45134fed9d/manager/1.log" Jul 06 20:47:00.082966 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:00.082944 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-cdjxm_6c277f91-c521-4062-bb13-d23bdeb073aa/limitador/0.log" Jul 06 20:47:00.205824 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:00.205800 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2k7bg_fb33694d-e0bd-479a-89cc-1cdc2be67151/manager/1.log" Jul 06 20:47:00.912940 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:00.912911 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64f5c756bf-p8brb_7678ddd9-44b2-49b0-9bc2-da640345aea2/kube-auth-proxy/0.log" Jul 06 20:47:01.033506 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:01.033465 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jxm4s_1dd007b4-41d0-46a4-ba88-5ee40d497497/istio-proxy/1.log" Jul 06 20:47:01.385046 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:01.385016 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d5cf567d5-sqrwr_efe2b855-7351-46d1-8cc7-a7d96ff57e80/router/0.log" Jul 06 20:47:06.980375 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.980341 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rsttk/must-gather-qfwx7"] Jul 06 20:47:06.980736 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.980719 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04da2b6d-76e5-40d3-bd6e-cbb446a56eb6" containerName="cleanup" Jul 06 20:47:06.980736 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.980729 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="04da2b6d-76e5-40d3-bd6e-cbb446a56eb6" containerName="cleanup" Jul 06 20:47:06.980805 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.980743 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19716d30-2da7-4906-9bc9-942188dfa4e1" containerName="istio-proxy" Jul 06 20:47:06.980805 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.980749 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="19716d30-2da7-4906-9bc9-942188dfa4e1" containerName="istio-proxy" Jul 06 20:47:06.980805 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.980767 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bf5f93b-f441-415d-bb46-b87d5f4ecc1c" containerName="cleanup" Jul 06 20:47:06.980805 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.980773 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf5f93b-f441-415d-bb46-b87d5f4ecc1c" containerName="cleanup" Jul 06 20:47:06.980953 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.980837 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="04da2b6d-76e5-40d3-bd6e-cbb446a56eb6" containerName="cleanup" Jul 06 20:47:06.980953 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.980846 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="19716d30-2da7-4906-9bc9-942188dfa4e1" containerName="istio-proxy" Jul 06 20:47:06.980953 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.980852 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="19716d30-2da7-4906-9bc9-942188dfa4e1" containerName="istio-proxy" Jul 06 20:47:06.980953 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.980861 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bf5f93b-f441-415d-bb46-b87d5f4ecc1c" containerName="cleanup" Jul 06 20:47:06.980953 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.980935 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19716d30-2da7-4906-9bc9-942188dfa4e1" containerName="istio-proxy" Jul 06 20:47:06.980953 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.980941 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="19716d30-2da7-4906-9bc9-942188dfa4e1" containerName="istio-proxy" Jul 06 20:47:06.984278 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.984263 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rsttk/must-gather-qfwx7" Jul 06 20:47:06.986559 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.986535 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rsttk\"/\"kube-root-ca.crt\"" Jul 06 20:47:06.987480 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.987454 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rsttk\"/\"default-dockercfg-tvshg\"" Jul 06 20:47:06.987578 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.987489 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rsttk\"/\"openshift-service-ca.crt\"" Jul 06 20:47:06.993438 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:06.993420 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rsttk/must-gather-qfwx7"] Jul 06 20:47:07.080877 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:07.080853 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f09822d6-d65b-419c-b473-12568a9a0029-must-gather-output\") pod \"must-gather-qfwx7\" (UID: \"f09822d6-d65b-419c-b473-12568a9a0029\") " pod="openshift-must-gather-rsttk/must-gather-qfwx7" Jul 06 20:47:07.081028 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:07.081009 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs2gf\" (UniqueName: \"kubernetes.io/projected/f09822d6-d65b-419c-b473-12568a9a0029-kube-api-access-hs2gf\") pod \"must-gather-qfwx7\" (UID: \"f09822d6-d65b-419c-b473-12568a9a0029\") " pod="openshift-must-gather-rsttk/must-gather-qfwx7" Jul 06 20:47:07.181518 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:07.181495 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs2gf\" (UniqueName: \"kubernetes.io/projected/f09822d6-d65b-419c-b473-12568a9a0029-kube-api-access-hs2gf\") pod \"must-gather-qfwx7\" (UID: \"f09822d6-d65b-419c-b473-12568a9a0029\") " pod="openshift-must-gather-rsttk/must-gather-qfwx7" Jul 06 20:47:07.181642 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:07.181531 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f09822d6-d65b-419c-b473-12568a9a0029-must-gather-output\") pod \"must-gather-qfwx7\" (UID: \"f09822d6-d65b-419c-b473-12568a9a0029\") " pod="openshift-must-gather-rsttk/must-gather-qfwx7" Jul 06 20:47:07.181800 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:07.181782 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f09822d6-d65b-419c-b473-12568a9a0029-must-gather-output\") pod \"must-gather-qfwx7\" (UID: \"f09822d6-d65b-419c-b473-12568a9a0029\") " pod="openshift-must-gather-rsttk/must-gather-qfwx7" Jul 06 20:47:07.192696 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:07.192672 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs2gf\" (UniqueName: \"kubernetes.io/projected/f09822d6-d65b-419c-b473-12568a9a0029-kube-api-access-hs2gf\") pod \"must-gather-qfwx7\" (UID: \"f09822d6-d65b-419c-b473-12568a9a0029\") " pod="openshift-must-gather-rsttk/must-gather-qfwx7" Jul 06 20:47:07.294018 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:07.293966 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rsttk/must-gather-qfwx7" Jul 06 20:47:07.429220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:07.429192 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rsttk/must-gather-qfwx7"] Jul 06 20:47:07.430793 ip-10-0-130-133 kubenswrapper[2583]: W0706 20:47:07.430765 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf09822d6_d65b_419c_b473_12568a9a0029.slice/crio-16898da8cdfd7a83dc021ae3d51aad65c623c1d8faa4303bd4166d5929931275 WatchSource:0}: Error finding container 16898da8cdfd7a83dc021ae3d51aad65c623c1d8faa4303bd4166d5929931275: Status 404 returned error can't find the container with id 16898da8cdfd7a83dc021ae3d51aad65c623c1d8faa4303bd4166d5929931275 Jul 06 20:47:07.432683 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:07.432665 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 20:47:08.105313 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:08.105270 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rsttk/must-gather-qfwx7" event={"ID":"f09822d6-d65b-419c-b473-12568a9a0029","Type":"ContainerStarted","Data":"16898da8cdfd7a83dc021ae3d51aad65c623c1d8faa4303bd4166d5929931275"} Jul 06 20:47:09.116692 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:09.116648 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rsttk/must-gather-qfwx7" event={"ID":"f09822d6-d65b-419c-b473-12568a9a0029","Type":"ContainerStarted","Data":"94a0c7fdc5cfd243926706e635472a776d740ec8eb44c0104870266338ca2ccf"} Jul 06 20:47:09.116692 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:09.116697 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rsttk/must-gather-qfwx7" event={"ID":"f09822d6-d65b-419c-b473-12568a9a0029","Type":"ContainerStarted","Data":"e3ba172e062f69ea48c44596795bf4916048019b075abcbdcf4ffcb4e81e6a0e"} Jul 06 20:47:09.135212 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:09.135162 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rsttk/must-gather-qfwx7" podStartSLOduration=2.254275353 podStartE2EDuration="3.135146844s" totalImagesPullingTime="880.871491ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:47:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:47:09.134085468 +0000 UTC m=+2482.260997573" watchObservedRunningTime="2026-07-06 20:47:09.135146844 +0000 UTC m=+2482.262058948" Jul 06 20:47:09.963415 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:09.963383 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-r6ptl_0b98a719-eb47-4991-93e9-271e9a4f0307/global-pull-secret-syncer/0.log" Jul 06 20:47:10.133231 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:10.133163 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wvqlx_85293479-2c8e-4065-9ec5-7f5f44b853fb/konnectivity-agent/0.log" Jul 06 20:47:10.157207 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:10.157181 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-133.ec2.internal_19411eae83bc603ba88b7f2105ff736e/haproxy/0.log" Jul 06 20:47:14.630233 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:14.630185 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd_5884ee3f-4633-4425-a422-abcb819e01b5/extract/0.log" Jul 06 20:47:14.659496 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:14.659457 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd_5884ee3f-4633-4425-a422-abcb819e01b5/util/0.log" Jul 06 20:47:14.685495 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:14.685437 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ptszd_5884ee3f-4633-4425-a422-abcb819e01b5/pull/0.log" Jul 06 20:47:14.715318 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:14.715255 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6_6603e100-ee5e-4c3d-9063-0b638e10b392/extract/0.log" Jul 06 20:47:14.743305 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:14.743226 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6_6603e100-ee5e-4c3d-9063-0b638e10b392/util/0.log" Jul 06 20:47:14.772260 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:14.772233 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04lbm6_6603e100-ee5e-4c3d-9063-0b638e10b392/pull/0.log" Jul 06 20:47:14.821305 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:14.821199 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96_6485c000-68a5-4ee0-a899-9925bee9b133/extract/0.log" Jul 06 20:47:14.846594 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:14.846570 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96_6485c000-68a5-4ee0-a899-9925bee9b133/util/0.log" Jul 06 20:47:14.871325 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:14.871301 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73l6l96_6485c000-68a5-4ee0-a899-9925bee9b133/pull/0.log" Jul 06 20:47:14.905064 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:14.904982 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5_602cda8d-f8c4-4a6a-9a04-5759fc1af23c/extract/0.log" Jul 06 20:47:14.928849 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:14.928824 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5_602cda8d-f8c4-4a6a-9a04-5759fc1af23c/util/0.log" Jul 06 20:47:14.958442 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:14.958415 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lvbv5_602cda8d-f8c4-4a6a-9a04-5759fc1af23c/pull/0.log" Jul 06 20:47:15.370894 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:15.369947 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5d5958db9b-tfpxl_b271f5aa-0263-47ea-9f03-b2dcbc80a78e/authorino/0.log" Jul 06 20:47:15.398569 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:15.398523 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tj6k6_ba75fdd1-3a5a-46cb-a655-a46013cf495e/manager/1.log" Jul 06 20:47:15.402041 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:15.401969 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tj6k6_ba75fdd1-3a5a-46cb-a655-a46013cf495e/manager/0.log" Jul 06 20:47:15.429380 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:15.429351 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-drtjk_0732e11e-ca32-46dc-b9fe-a7ad6ef72eae/manager/1.log" Jul 06 20:47:15.430335 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:15.430309 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-drtjk_0732e11e-ca32-46dc-b9fe-a7ad6ef72eae/manager/0.log" Jul 06 20:47:15.455663 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:15.455639 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-b5j27_7e2f1dd2-f940-4c42-89f3-e8691ec33679/kuadrant-console-plugin/0.log" Jul 06 20:47:15.511676 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:15.511559 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4khpv_d4d809a6-2593-4470-b8a9-49c663b07b0e/registry-server/0.log" Jul 06 20:47:15.678721 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:15.678627 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-pzfd9_a9399ca9-ef00-49bc-97f9-ce45134fed9d/manager/1.log" Jul 06 20:47:15.682489 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:15.682457 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-pzfd9_a9399ca9-ef00-49bc-97f9-ce45134fed9d/manager/0.log" Jul 06 20:47:15.717725 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:15.717697 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-cdjxm_6c277f91-c521-4062-bb13-d23bdeb073aa/limitador/0.log" Jul 06 20:47:15.803892 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:15.803102 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2k7bg_fb33694d-e0bd-479a-89cc-1cdc2be67151/manager/1.log" Jul 06 20:47:15.813220 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:15.811241 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2k7bg_fb33694d-e0bd-479a-89cc-1cdc2be67151/manager/0.log" Jul 06 20:47:17.513714 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:17.513684 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5f641b0d-9b3c-49a9-8455-6818dd958c8f/alertmanager/0.log" Jul 06 20:47:17.538707 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:17.538675 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5f641b0d-9b3c-49a9-8455-6818dd958c8f/config-reloader/0.log" Jul 06 20:47:17.565126 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:17.565100 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5f641b0d-9b3c-49a9-8455-6818dd958c8f/kube-rbac-proxy-web/0.log" Jul 06 20:47:17.589439 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:17.589410 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5f641b0d-9b3c-49a9-8455-6818dd958c8f/kube-rbac-proxy/0.log" Jul 06 20:47:17.619396 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:17.619322 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5f641b0d-9b3c-49a9-8455-6818dd958c8f/kube-rbac-proxy-metric/0.log" Jul 06 20:47:17.643675 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:17.643650 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5f641b0d-9b3c-49a9-8455-6818dd958c8f/prom-label-proxy/0.log" Jul 06 20:47:17.670035 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:17.669982 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5f641b0d-9b3c-49a9-8455-6818dd958c8f/init-config-reloader/0.log" Jul 06 20:47:17.713769 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:17.713739 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-mvwbg_7f52f514-81ea-40b8-b6aa-9ab51d3f3462/cluster-monitoring-operator/0.log" Jul 06 20:47:17.748067 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:17.748035 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-vpzgl_19582221-42b6-43fe-86ac-c5f1f2aa9e77/kube-state-metrics/0.log" Jul 06 20:47:17.769589 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:17.769509 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-vpzgl_19582221-42b6-43fe-86ac-c5f1f2aa9e77/kube-rbac-proxy-main/0.log" Jul 06 20:47:17.797622 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:17.797578 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-vpzgl_19582221-42b6-43fe-86ac-c5f1f2aa9e77/kube-rbac-proxy-self/0.log" Jul 06 20:47:17.992327 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:17.992297 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nsnl5_41eb1a6e-35a2-4a3b-8459-06ad50a80756/node-exporter/0.log" Jul 06 20:47:18.016203 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:18.016181 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nsnl5_41eb1a6e-35a2-4a3b-8459-06ad50a80756/kube-rbac-proxy/0.log" Jul 06 20:47:18.043391 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:18.043316 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nsnl5_41eb1a6e-35a2-4a3b-8459-06ad50a80756/init-textfile/0.log" Jul 06 20:47:18.168044 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:18.167993 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-dmrp2_caaba1f6-5de3-45d3-bd70-c71692fd5877/kube-rbac-proxy-main/0.log" Jul 06 20:47:18.193954 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:18.193917 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-dmrp2_caaba1f6-5de3-45d3-bd70-c71692fd5877/kube-rbac-proxy-self/0.log" Jul 06 20:47:18.220154 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:18.220126 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-dmrp2_caaba1f6-5de3-45d3-bd70-c71692fd5877/openshift-state-metrics/0.log" Jul 06 20:47:19.694741 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.694695 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6"] Jul 06 20:47:19.703165 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.703133 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.707011 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.706981 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6"] Jul 06 20:47:19.854837 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.854803 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f7673c7-1f5f-48c6-b3c5-d9828f759697-podres\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.855043 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.854946 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f7673c7-1f5f-48c6-b3c5-d9828f759697-sys\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.855043 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.855002 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f7673c7-1f5f-48c6-b3c5-d9828f759697-proc\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.855165 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.855050 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n69cz\" (UniqueName: \"kubernetes.io/projected/5f7673c7-1f5f-48c6-b3c5-d9828f759697-kube-api-access-n69cz\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.855165 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.855153 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f7673c7-1f5f-48c6-b3c5-d9828f759697-lib-modules\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.888165 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.888135 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-8s7zn_a39e36ae-32ec-4151-93b5-bb6fa2b01a31/networking-console-plugin/0.log" Jul 06 20:47:19.955772 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.955686 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f7673c7-1f5f-48c6-b3c5-d9828f759697-sys\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.956035 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.956010 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f7673c7-1f5f-48c6-b3c5-d9828f759697-proc\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.956161 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.956145 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n69cz\" (UniqueName: \"kubernetes.io/projected/5f7673c7-1f5f-48c6-b3c5-d9828f759697-kube-api-access-n69cz\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.956327 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.956304 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f7673c7-1f5f-48c6-b3c5-d9828f759697-lib-modules\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.956458 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.956438 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f7673c7-1f5f-48c6-b3c5-d9828f759697-sys\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.956524 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.956495 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5f7673c7-1f5f-48c6-b3c5-d9828f759697-proc\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.956733 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.956713 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f7673c7-1f5f-48c6-b3c5-d9828f759697-podres\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.956807 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.956733 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f7673c7-1f5f-48c6-b3c5-d9828f759697-lib-modules\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.956890 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.956831 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5f7673c7-1f5f-48c6-b3c5-d9828f759697-podres\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:19.966069 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:19.966035 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n69cz\" (UniqueName: \"kubernetes.io/projected/5f7673c7-1f5f-48c6-b3c5-d9828f759697-kube-api-access-n69cz\") pod \"perf-node-gather-daemonset-txhf6\" (UID: \"5f7673c7-1f5f-48c6-b3c5-d9828f759697\") " pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:20.018250 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:20.018217 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:20.218170 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:20.218087 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6"] Jul 06 20:47:20.541174 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:20.541142 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/1.log" Jul 06 20:47:20.548435 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:20.548409 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-dddkw_0962b7d0-5a52-4745-a3c3-65afa1f24ca1/console-operator/2.log" Jul 06 20:47:21.072069 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:21.072035 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7cbcdfd7f9-qh25x_33f8fb7e-4604-40f9-a874-6154912fcbaf/console/0.log" Jul 06 20:47:21.226455 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:21.226409 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" event={"ID":"5f7673c7-1f5f-48c6-b3c5-d9828f759697","Type":"ContainerStarted","Data":"90f360beb9810d7d3d4c93b21f4cb2118cd5e618ed9afea6a1ef0652f07adb8d"} Jul 06 20:47:21.226644 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:21.226471 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" event={"ID":"5f7673c7-1f5f-48c6-b3c5-d9828f759697","Type":"ContainerStarted","Data":"54c7fa8027f58bdb87f002ed5300cce62cfab9fc19514690f27ae83f1a92888b"} Jul 06 20:47:21.226644 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:21.226506 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:21.245943 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:21.245867 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" podStartSLOduration=2.245852464 podStartE2EDuration="2.245852464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:47:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:47:21.244322444 +0000 UTC m=+2494.371234549" watchObservedRunningTime="2026-07-06 20:47:21.245852464 +0000 UTC m=+2494.372764567" Jul 06 20:47:21.711243 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:21.711214 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-ppjpz_f69be79e-9600-4313-8c0e-88348a100a7c/volume-data-source-validator/0.log" Jul 06 20:47:22.792653 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:22.792625 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nzdbq_ce43eb8c-4158-49b9-867a-900edcdd00c9/dns/0.log" Jul 06 20:47:22.819934 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:22.819903 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nzdbq_ce43eb8c-4158-49b9-867a-900edcdd00c9/kube-rbac-proxy/0.log" Jul 06 20:47:23.015692 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:23.015661 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mpdph_0fcc19e6-df9c-4444-981c-898f00976382/dns-node-resolver/0.log" Jul 06 20:47:23.651636 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:23.651589 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wm5jd_36bc4e9d-a829-4f2a-906e-754c4cef2a5d/node-ca/0.log" Jul 06 20:47:24.792228 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:24.792199 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64f5c756bf-p8brb_7678ddd9-44b2-49b0-9bc2-da640345aea2/kube-auth-proxy/0.log" Jul 06 20:47:24.823308 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:24.823275 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jxm4s_1dd007b4-41d0-46a4-ba88-5ee40d497497/istio-proxy/0.log" Jul 06 20:47:24.858769 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:24.858718 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jxm4s_1dd007b4-41d0-46a4-ba88-5ee40d497497/istio-proxy/1.log" Jul 06 20:47:24.984069 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:24.984032 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d5cf567d5-sqrwr_efe2b855-7351-46d1-8cc7-a7d96ff57e80/router/0.log" Jul 06 20:47:25.730175 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:25.730137 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7qq9j_59096fca-4e28-4504-aaa4-f99361c31d44/serve-healthcheck-canary/0.log" Jul 06 20:47:26.236967 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:26.236938 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-54f2g_db968f0a-d281-4b2b-8eaf-285d77c876b2/insights-operator/0.log" Jul 06 20:47:26.237485 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:26.237408 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-54f2g_db968f0a-d281-4b2b-8eaf-285d77c876b2/insights-operator/1.log" Jul 06 20:47:26.332680 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:26.332647 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kzg95_e95ceb16-8f18-43d5-8a9c-fce4e159bb0f/kube-rbac-proxy/0.log" Jul 06 20:47:26.355974 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:26.355945 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kzg95_e95ceb16-8f18-43d5-8a9c-fce4e159bb0f/exporter/0.log" Jul 06 20:47:26.381944 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:26.381920 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kzg95_e95ceb16-8f18-43d5-8a9c-fce4e159bb0f/extractor/0.log" Jul 06 20:47:27.247061 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:27.247032 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rsttk/perf-node-gather-daemonset-txhf6" Jul 06 20:47:28.863462 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:28.863373 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bfvds_73340a04-d613-40d5-85ab-7c54daeaa757/manager/1.log" Jul 06 20:47:28.863961 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:28.863549 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bfvds_73340a04-d613-40d5-85ab-7c54daeaa757/manager/0.log" Jul 06 20:47:28.994154 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:28.994087 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7vpbr_4c49034e-79c9-407b-b068-96bbe8d61dac/manager/0.log" Jul 06 20:47:29.021250 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:29.021219 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7vpbr_4c49034e-79c9-407b-b068-96bbe8d61dac/manager/1.log" Jul 06 20:47:29.208595 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:29.208514 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-np686_ed7b2a0f-049e-4874-8a09-076638035822/manager/1.log" Jul 06 20:47:29.209408 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:29.209381 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-np686_ed7b2a0f-049e-4874-8a09-076638035822/manager/0.log" Jul 06 20:47:29.378938 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:29.378843 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cbf59454-jvfwm_a5f8a799-9d7a-498c-8e6d-8baa3036efab/manager/1.log" Jul 06 20:47:29.429722 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:29.429684 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cbf59454-jvfwm_a5f8a799-9d7a-498c-8e6d-8baa3036efab/manager/0.log" Jul 06 20:47:29.451129 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:29.451102 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cbf59454-jvfwm_a5f8a799-9d7a-498c-8e6d-8baa3036efab/copy-manifests/0.log" Jul 06 20:47:30.134108 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:30.134080 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64f5c756bf-p8brb_7678ddd9-44b2-49b0-9bc2-da640345aea2/kube-auth-proxy/0.log" Jul 06 20:47:30.153643 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:30.153615 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jxm4s_1dd007b4-41d0-46a4-ba88-5ee40d497497/istio-proxy/0.log" Jul 06 20:47:30.186817 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:30.186790 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jxm4s_1dd007b4-41d0-46a4-ba88-5ee40d497497/istio-proxy/1.log" Jul 06 20:47:30.261375 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:30.261341 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d5cf567d5-sqrwr_efe2b855-7351-46d1-8cc7-a7d96ff57e80/router/0.log" Jul 06 20:47:30.803524 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:30.803489 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-hlztr_87e2e689-5dd8-4f87-ab4b-8a5d8f3096bd/openshift-lws-operator/0.log" Jul 06 20:47:31.752816 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:31.752758 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64f5c756bf-p8brb_7678ddd9-44b2-49b0-9bc2-da640345aea2/kube-auth-proxy/0.log" Jul 06 20:47:31.774094 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:31.774063 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jxm4s_1dd007b4-41d0-46a4-ba88-5ee40d497497/istio-proxy/0.log" Jul 06 20:47:31.805885 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:31.805845 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jxm4s_1dd007b4-41d0-46a4-ba88-5ee40d497497/istio-proxy/1.log" Jul 06 20:47:31.883727 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:31.883651 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d5cf567d5-sqrwr_efe2b855-7351-46d1-8cc7-a7d96ff57e80/router/0.log" Jul 06 20:47:36.193439 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:36.193385 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-hbc8g_b359f2f9-4eb6-47f8-90cf-f096ba41e852/kube-storage-version-migrator-operator/1.log" Jul 06 20:47:36.194284 ip-10-0-130-133 kubenswrapper[2583]: I0706 20:47:36.194263 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-hbc8g_b359f2f9-4eb6-47f8-90cf-f096ba41e852/kube-storage-version-migrator-operator/0.log"