Jul 03 20:30:18.220949 ip-10-0-139-68 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 03 20:30:18.220960 ip-10-0-139-68 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 03 20:30:18.220966 ip-10-0-139-68 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 03 20:30:18.221229 ip-10-0-139-68 systemd[1]: Failed to start Kubernetes Kubelet. Jul 03 20:30:27.660567 ip-10-0-139-68 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 03 20:30:27.660579 ip-10-0-139-68 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 1b796313071d41048c1c809dc9d8524c -- Jul 03 20:32:13.437070 ip-10-0-139-68 systemd[1]: Starting Kubernetes Kubelet... Jul 03 20:32:13.791840 ip-10-0-139-68 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 20:32:13.791840 ip-10-0-139-68 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 03 20:32:13.791840 ip-10-0-139-68 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 20:32:13.791840 ip-10-0-139-68 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 03 20:32:13.791840 ip-10-0-139-68 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 20:32:13.792799 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.792631 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 03 20:32:13.797231 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797210 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 20:32:13.797231 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797231 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797236 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797239 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797243 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797246 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797250 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797252 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797255 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797258 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797261 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797264 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797272 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797274 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797277 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797280 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797282 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797284 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797287 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797289 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797292 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 20:32:13.797303 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797295 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797297 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797300 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797303 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797305 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797308 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797311 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797313 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797316 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797320 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797322 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797334 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797337 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797339 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797342 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797345 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797347 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797349 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797352 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797355 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 20:32:13.797841 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797357 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797359 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797362 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797364 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797367 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797369 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797372 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797374 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797376 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797379 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797381 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797383 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797386 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797389 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797392 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797395 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797398 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797400 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797402 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797405 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 03 20:32:13.798328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797408 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797412 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797416 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797418 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797421 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797424 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797426 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797429 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797431 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797433 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797436 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797438 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797440 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797443 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797445 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797449 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797453 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797456 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797459 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797461 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 20:32:13.798828 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797464 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797466 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797468 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797471 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797473 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797476 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797479 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797481 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797484 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797486 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797488 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797491 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797494 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797496 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797498 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797501 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797503 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.797507 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798228 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798234 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 20:32:13.799321 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798237 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798240 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798242 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798245 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798250 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798254 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798257 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798260 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798262 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798265 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798267 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798270 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798272 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798275 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798277 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798279 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798282 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798285 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798288 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 20:32:13.799822 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798290 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798293 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798296 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798300 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798303 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798306 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798308 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798311 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798314 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798316 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798318 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798321 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798324 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798326 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798329 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798332 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798334 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798337 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798339 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 20:32:13.800294 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798342 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798344 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798347 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798349 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798352 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798354 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798357 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798359 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798361 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798363 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798366 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798370 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798373 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798376 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798378 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798380 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798383 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798385 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798387 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798390 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 20:32:13.800766 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798392 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798395 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798398 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798401 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798403 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798407 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798409 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798412 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798414 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798416 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798419 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798421 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798424 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798426 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798429 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798431 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798434 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798436 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798438 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798441 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 20:32:13.801269 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798443 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798446 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798448 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798455 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798458 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798462 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798466 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798469 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798472 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798475 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798479 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798482 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798485 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798488 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798490 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798493 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798495 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798498 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798501 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 20:32:13.801809 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798907 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798913 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798917 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798920 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798923 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798926 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798928 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798931 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798935 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798938 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798941 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798943 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798946 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798949 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798951 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798954 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798959 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798963 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798965 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 20:32:13.802276 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798968 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798971 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798973 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798976 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798978 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798981 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798983 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798986 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798988 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798991 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798993 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798996 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.798999 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799002 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799005 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799007 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799009 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799013 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799015 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799018 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 20:32:13.802791 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799020 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799023 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799025 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799027 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799030 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799032 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799035 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799037 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799040 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799044 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799047 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799049 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799052 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799054 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799057 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799060 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799063 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799065 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799067 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799070 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 20:32:13.803365 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799072 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799075 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799077 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799080 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799082 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799085 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799087 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799090 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799092 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799095 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799097 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799100 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799102 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799104 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799107 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799109 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799112 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799114 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799117 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799119 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 20:32:13.803882 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799121 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799125 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799128 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799131 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799133 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799136 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799138 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799141 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799143 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799146 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799148 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799151 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799153 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799156 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799158 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799160 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799163 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799165 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799168 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799171 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 20:32:13.804395 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799245 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799261 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799270 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799276 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799283 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799288 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799292 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799296 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799299 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799302 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799306 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799316 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799320 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799325 2582 flags.go:64] FLAG: --cgroup-root="" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799328 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799331 2582 flags.go:64] FLAG: --client-ca-file="" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799334 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799337 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799342 2582 flags.go:64] FLAG: --cluster-domain="" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799345 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799348 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799351 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799354 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799358 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 03 20:32:13.804889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799361 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799364 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799367 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799370 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799373 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799376 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799379 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799383 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799387 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799390 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799393 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799396 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799398 2582 flags.go:64] FLAG: --enable-server="true" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799401 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799405 2582 flags.go:64] FLAG: --event-burst="100" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799408 2582 flags.go:64] FLAG: --event-qps="50" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799411 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799414 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799417 2582 flags.go:64] FLAG: --eviction-hard="" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799420 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799423 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799428 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799431 2582 flags.go:64] FLAG: --eviction-soft="" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799433 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799436 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 03 20:32:13.805475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799439 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799442 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799445 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799448 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799451 2582 flags.go:64] FLAG: --feature-gates="" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799455 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799457 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799461 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799464 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799467 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799469 2582 flags.go:64] FLAG: --help="false" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799472 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-139-68.ec2.internal" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799475 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799478 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799481 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799485 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799488 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799490 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799493 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799496 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799498 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799501 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799503 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799506 2582 flags.go:64] FLAG: --kube-reserved="" Jul 03 20:32:13.806155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799509 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799511 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799514 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799517 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799521 2582 flags.go:64] FLAG: --lock-file="" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799524 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799526 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799529 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799535 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799537 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799540 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799543 2582 flags.go:64] FLAG: --logging-format="text" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799546 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799549 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799553 2582 flags.go:64] FLAG: --manifest-url="" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799555 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799560 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799563 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799567 2582 flags.go:64] FLAG: --max-pods="110" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799569 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799572 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799575 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799578 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799581 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799584 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 03 20:32:13.806801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799587 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799593 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799596 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799599 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799602 2582 flags.go:64] FLAG: --pod-cidr="" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799605 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799610 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799613 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799616 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799619 2582 flags.go:64] FLAG: --port="10250" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799622 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799624 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02397a4aa5452220d" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799629 2582 flags.go:64] FLAG: --qos-reserved="" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799631 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799634 2582 flags.go:64] FLAG: --register-node="true" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799637 2582 flags.go:64] FLAG: --register-with-taints="" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799641 2582 flags.go:64] FLAG: --registry-burst="10" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799644 2582 flags.go:64] FLAG: --registry-qps="5" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799647 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799649 2582 flags.go:64] FLAG: --reserved-memory="" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799653 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799656 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799658 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799661 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799664 2582 flags.go:64] FLAG: --runonce="false" Jul 03 20:32:13.807424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799666 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799669 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799672 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799675 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799678 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799680 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799683 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799687 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799690 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799692 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799695 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799697 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799700 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799703 2582 flags.go:64] FLAG: --system-cgroups="" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799706 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799711 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799714 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799716 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799721 2582 flags.go:64] FLAG: --tls-min-version="" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799725 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799728 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799731 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799733 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799736 2582 flags.go:64] FLAG: --v="2" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799740 2582 flags.go:64] FLAG: --version="false" Jul 03 20:32:13.808329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799758 2582 flags.go:64] FLAG: --vmodule="" Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799762 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.799765 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799883 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799887 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799891 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799894 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799897 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799899 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799902 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799905 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799908 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799911 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799914 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799916 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799920 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799922 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799925 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799927 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799930 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 20:32:13.809298 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799932 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799934 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799937 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799939 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799941 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799944 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799947 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799949 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799952 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799954 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799957 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799959 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799963 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799966 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799968 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799971 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799973 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799975 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799979 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 20:32:13.809846 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799981 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799983 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799986 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799988 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799991 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799993 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799996 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.799998 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800001 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800005 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800008 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800010 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800013 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800016 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800018 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800021 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800023 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800026 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800028 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 20:32:13.810313 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800031 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800034 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800036 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800038 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800041 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800044 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800048 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800051 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800053 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800056 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800058 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800061 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800063 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800065 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800068 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800070 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800073 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800075 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800078 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800080 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 20:32:13.810801 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800082 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800085 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800088 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800092 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800094 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800097 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800099 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800102 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800104 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800106 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800108 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800111 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800114 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800117 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800120 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800123 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800126 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800129 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800132 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 20:32:13.811289 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800135 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800138 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800141 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800143 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.800146 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.800840 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.809331 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.809349 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.809370 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.809379 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809440 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809445 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809448 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809451 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809455 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 20:32:13.811765 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809458 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809461 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809464 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809467 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809470 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809472 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809475 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809477 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809480 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809483 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809485 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809488 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809491 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809493 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809496 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809499 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809501 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809504 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809506 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809509 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 20:32:13.812153 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809511 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809514 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809516 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809519 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809523 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809525 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809527 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809530 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809532 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809536 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809538 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809541 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809543 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809546 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809548 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809551 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809553 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809556 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809558 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809561 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 20:32:13.812697 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809563 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809567 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809570 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809574 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809577 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809580 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809582 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809585 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809588 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809591 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809593 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809596 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809598 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809601 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809604 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809606 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809609 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809612 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809615 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 20:32:13.813211 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809617 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809620 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809622 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809624 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809627 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809630 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809632 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809635 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809637 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809640 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809642 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809645 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809647 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809651 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809654 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809658 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809660 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809663 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809667 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 20:32:13.813670 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809670 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809673 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809676 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809679 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809681 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809684 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809687 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809690 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809693 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809695 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809698 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809701 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809704 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809706 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809709 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 20:32:13.814292 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809712 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.809717 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809853 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809860 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809863 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809866 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809869 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809872 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809877 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809880 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809882 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809885 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809887 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809890 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809893 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 20:32:13.814660 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809895 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809898 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809900 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809903 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809906 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809909 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809911 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809914 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809916 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809919 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809921 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809923 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809926 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809929 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809931 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809934 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809936 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809939 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809941 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809944 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 20:32:13.815053 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809946 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809949 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809951 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809954 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809956 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809958 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809961 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809963 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809965 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809968 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809970 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809972 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809976 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809978 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809981 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809983 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809985 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809988 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809990 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 20:32:13.815545 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809993 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809995 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.809998 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810000 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810002 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810005 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810008 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810010 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810013 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810015 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810018 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810020 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810022 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810025 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810027 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810030 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810032 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810035 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810037 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810039 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 20:32:13.816040 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810042 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810044 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810047 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810049 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810052 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810054 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810058 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810061 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810063 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810066 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810068 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810071 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810073 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810075 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810078 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810080 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810083 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810085 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810088 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810090 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 20:32:13.816529 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810093 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 20:32:13.817018 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810096 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 20:32:13.817018 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810099 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 20:32:13.817018 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810102 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 20:32:13.817018 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810105 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 20:32:13.817018 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810108 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 20:32:13.817018 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:13.810111 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 20:32:13.817018 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.810116 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 20:32:13.817018 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.810241 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 03 20:32:13.817018 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.812228 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 03 20:32:13.817018 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.813533 2582 server.go:1019] "Starting client certificate rotation" Jul 03 20:32:13.817018 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.813639 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 03 20:32:13.817018 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.813674 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 03 20:32:13.817315 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.816156 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 03 20:32:13.817482 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.817461 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 03 20:32:13.827494 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.827470 2582 log.go:25] "Validated CRI v1 runtime API" Jul 03 20:32:13.830677 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.830658 2582 log.go:25] "Validated CRI v1 image API" Jul 03 20:32:13.832179 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.832162 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 03 20:32:13.834344 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.834324 2582 fs.go:135] Filesystem UUIDs: map[28ac1d76-ef9d-4a8e-b7d5-88f4d41eaec3:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b615593c-b572-4150-b3d6-40d22e4ead3d:/dev/nvme0n1p3] Jul 03 20:32:13.834412 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.834344 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 03 20:32:13.840445 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.840296 2582 manager.go:216] Machine: {Timestamp:2026-07-03 20:32:13.839014482 +0000 UTC m=+0.297286913 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098907 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec279e0833a99342bec42290c98e62fc SystemUUID:ec279e08-33a9-9342-bec4-2290c98e62fc BootID:1b796313-071d-4104-8c1c-809dc9d8524c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:04:e4:b7:1e:8d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:04:e4:b7:1e:8d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:a3:3c:d8:39:dd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 03 20:32:13.840445 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.840442 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 03 20:32:13.840572 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.840550 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 03 20:32:13.843292 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.843258 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 03 20:32:13.843491 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.843475 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 03 20:32:13.843522 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.843302 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-68.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 03 20:32:13.843981 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.843968 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 03 20:32:13.844018 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.843986 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 03 20:32:13.844018 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.843995 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 03 20:32:13.844592 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.844582 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 03 20:32:13.845778 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.845767 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 03 20:32:13.848567 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.848553 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 03 20:32:13.848694 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.848685 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 03 20:32:13.848728 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.848719 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 03 20:32:13.848776 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.848729 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 03 20:32:13.848776 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.848762 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 03 20:32:13.848776 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.848775 2582 kubelet.go:398] "Adding apiserver pod source" Jul 03 20:32:13.848857 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.848785 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 03 20:32:13.849842 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.849816 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 03 20:32:13.849971 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.849851 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 03 20:32:13.852553 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.852533 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 03 20:32:13.854278 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.854265 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 03 20:32:13.854326 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.854295 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 03 20:32:13.855285 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.855273 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 03 20:32:13.855332 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.855291 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 03 20:32:13.855332 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.855297 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 03 20:32:13.855332 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.855303 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 03 20:32:13.855332 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.855320 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 03 20:32:13.855332 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.855326 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 03 20:32:13.855486 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.855335 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 03 20:32:13.855486 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.855341 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 03 20:32:13.855486 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.855348 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 03 20:32:13.855486 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.855354 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 03 20:32:13.855486 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.855363 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 03 20:32:13.855486 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.855368 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 03 20:32:13.856015 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.856002 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 03 20:32:13.856048 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.856017 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 03 20:32:13.859540 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.859524 2582 server.go:1268] "Started kubelet" Jul 03 20:32:13.859633 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.859593 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 03 20:32:13.859689 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.859615 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 03 20:32:13.859737 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.859701 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 03 20:32:13.860700 ip-10-0-139-68 systemd[1]: Started Kubernetes Kubelet. Jul 03 20:32:13.860840 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.860741 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-68.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 03 20:32:13.860840 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:13.860816 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 03 20:32:13.860959 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:13.860839 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 03 20:32:13.861961 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.861741 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 03 20:32:13.862381 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.862367 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 03 20:32:13.864422 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:13.864403 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 03 20:32:13.864938 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:13.864113 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-68.ec2.internal.18bee1c3dee1a6d6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-68.ec2.internal,UID:ip-10-0-139-68.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-68.ec2.internal,},FirstTimestamp:2026-07-03 20:32:13.859497686 +0000 UTC m=+0.317770116,LastTimestamp:2026-07-03 20:32:13.859497686 +0000 UTC m=+0.317770116,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-68.ec2.internal,}" Jul 03 20:32:13.865082 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.865066 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-675rs" Jul 03 20:32:13.869113 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.869094 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 03 20:32:13.869113 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.869107 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 03 20:32:13.869252 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:13.869186 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 03 20:32:13.869768 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.869733 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 03 20:32:13.869857 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.869818 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 03 20:32:13.869857 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.869832 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 03 20:32:13.869987 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.869972 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 03 20:32:13.869987 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.869987 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 03 20:32:13.870777 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.870737 2582 factory.go:153] Registering CRI-O factory Jul 03 20:32:13.870866 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.870823 2582 factory.go:223] Registration of the crio container factory successfully Jul 03 20:32:13.870920 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.870873 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 03 20:32:13.870920 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.870882 2582 factory.go:55] Registering systemd factory Jul 03 20:32:13.870920 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.870890 2582 factory.go:223] Registration of the systemd container factory successfully Jul 03 20:32:13.870920 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.870914 2582 factory.go:103] Registering Raw factory Jul 03 20:32:13.871091 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.870927 2582 manager.go:1196] Started watching for new ooms in manager Jul 03 20:32:13.871294 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.871280 2582 manager.go:349] Starting recovery of all containers Jul 03 20:32:13.873627 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.873605 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-675rs" Jul 03 20:32:13.880379 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.880352 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 20:32:13.880724 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.880544 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 03 20:32:13.883460 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.883440 2582 manager.go:354] Recovery completed Jul 03 20:32:13.883579 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:13.883446 2582 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-68.ec2.internal\" not found" node="ip-10-0-139-68.ec2.internal" Jul 03 20:32:13.887692 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.887680 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 20:32:13.889595 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.889576 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientMemory" Jul 03 20:32:13.889657 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.889613 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 20:32:13.889657 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.889627 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientPID" Jul 03 20:32:13.890200 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.890187 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 03 20:32:13.890200 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.890199 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 03 20:32:13.890294 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.890215 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 03 20:32:13.890294 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.890249 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 20:32:13.891407 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.891394 2582 policy_none.go:49] "None policy: Start" Jul 03 20:32:13.891466 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.891410 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 03 20:32:13.891466 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.891420 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 03 20:32:13.891616 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.891601 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientMemory" Jul 03 20:32:13.891676 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.891633 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 20:32:13.891676 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.891649 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientPID" Jul 03 20:32:13.892428 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.892416 2582 policy_none.go:47] "Start" Jul 03 20:32:13.937350 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.937334 2582 manager.go:337] "Starting Device Plugin manager" Jul 03 20:32:13.937471 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:13.937384 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 03 20:32:13.937471 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.937398 2582 server.go:87] "Starting device plugin registration server" Jul 03 20:32:13.937671 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.937656 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 03 20:32:13.937740 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.937673 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 03 20:32:13.937812 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.937769 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 03 20:32:13.937863 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.937855 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 03 20:32:13.937903 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.937864 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 03 20:32:13.938411 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:13.938391 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 03 20:32:13.938495 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:13.938437 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-68.ec2.internal\" not found" Jul 03 20:32:13.979684 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.979662 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 03 20:32:13.981574 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.979704 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 03 20:32:13.981574 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.979727 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 03 20:32:13.981574 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:13.979774 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 03 20:32:13.982095 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:13.982079 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 20:32:14.038211 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.038178 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 20:32:14.039176 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.039159 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientMemory" Jul 03 20:32:14.039251 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.039193 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 20:32:14.039251 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.039203 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientPID" Jul 03 20:32:14.039251 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.039228 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.047807 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.047763 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.047807 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:14.047786 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-68.ec2.internal\": node \"ip-10-0-139-68.ec2.internal\" not found" Jul 03 20:32:14.062266 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:14.062243 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 03 20:32:14.080519 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.080492 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal"] Jul 03 20:32:14.080575 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.080560 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 20:32:14.081469 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.081454 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientMemory" Jul 03 20:32:14.081530 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.081483 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 20:32:14.081530 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.081494 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientPID" Jul 03 20:32:14.082779 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.082765 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 20:32:14.082874 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.082856 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.082941 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.082892 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 20:32:14.083527 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.083513 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientMemory" Jul 03 20:32:14.083596 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.083538 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 20:32:14.083596 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.083548 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientPID" Jul 03 20:32:14.083596 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.083514 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientMemory" Jul 03 20:32:14.083684 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.083609 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 20:32:14.083684 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.083619 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientPID" Jul 03 20:32:14.084854 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.084833 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.084933 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.084867 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 20:32:14.085515 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.085502 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientMemory" Jul 03 20:32:14.085562 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.085529 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 20:32:14.085562 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.085540 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeHasSufficientPID" Jul 03 20:32:14.100311 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:14.100290 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-68.ec2.internal\" not found" node="ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.103760 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:14.103721 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-68.ec2.internal\" not found" node="ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.162480 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:14.162445 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 03 20:32:14.170933 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.170897 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6deb43b1e3d0667ae47de4e2350bbbcb-config\") pod \"kube-apiserver-proxy-ip-10-0-139-68.ec2.internal\" (UID: \"6deb43b1e3d0667ae47de4e2350bbbcb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.171049 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.170933 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/99a5d64bbd014c057347733a22ebc4d0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal\" (UID: \"99a5d64bbd014c057347733a22ebc4d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.171049 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.170959 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99a5d64bbd014c057347733a22ebc4d0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal\" (UID: \"99a5d64bbd014c057347733a22ebc4d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.262581 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:14.262539 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 03 20:32:14.271922 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.271878 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6deb43b1e3d0667ae47de4e2350bbbcb-config\") pod \"kube-apiserver-proxy-ip-10-0-139-68.ec2.internal\" (UID: \"6deb43b1e3d0667ae47de4e2350bbbcb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.271988 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.271935 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/99a5d64bbd014c057347733a22ebc4d0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal\" (UID: \"99a5d64bbd014c057347733a22ebc4d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.271988 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.271954 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99a5d64bbd014c057347733a22ebc4d0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal\" (UID: \"99a5d64bbd014c057347733a22ebc4d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.272056 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.271990 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99a5d64bbd014c057347733a22ebc4d0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal\" (UID: \"99a5d64bbd014c057347733a22ebc4d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.272056 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.271993 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/99a5d64bbd014c057347733a22ebc4d0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal\" (UID: \"99a5d64bbd014c057347733a22ebc4d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.272056 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.272004 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6deb43b1e3d0667ae47de4e2350bbbcb-config\") pod \"kube-apiserver-proxy-ip-10-0-139-68.ec2.internal\" (UID: \"6deb43b1e3d0667ae47de4e2350bbbcb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.363382 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:14.363311 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 03 20:32:14.403827 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.403804 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.406630 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.406601 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.463885 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:14.463855 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 03 20:32:14.564343 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:14.564319 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 03 20:32:14.664954 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:14.664877 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 03 20:32:14.738954 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.738924 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 20:32:14.765039 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:14.765007 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-68.ec2.internal\" not found" Jul 03 20:32:14.813060 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.813028 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 03 20:32:14.813622 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.813241 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 03 20:32:14.813622 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.813257 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 03 20:32:14.813622 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.813244 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 03 20:32:14.861035 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.861011 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 20:32:14.869849 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.869819 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.869849 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.869844 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 03 20:32:14.875688 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.875656 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-02 20:27:14 +0000 UTC" deadline="2028-04-02 01:06:52.926632034 +0000 UTC" Jul 03 20:32:14.875688 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.875684 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15316h34m38.050950634s" Jul 03 20:32:14.881950 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.881933 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 03 20:32:14.882862 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.882846 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" Jul 03 20:32:14.889531 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.889508 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 03 20:32:14.890974 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:14.890949 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6deb43b1e3d0667ae47de4e2350bbbcb.slice/crio-354180676d9357588e14748a202455251d72e0a42e2b9e725dfe53db839b3f09 WatchSource:0}: Error finding container 354180676d9357588e14748a202455251d72e0a42e2b9e725dfe53db839b3f09: Status 404 returned error can't find the container with id 354180676d9357588e14748a202455251d72e0a42e2b9e725dfe53db839b3f09 Jul 03 20:32:14.891554 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:14.891530 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a5d64bbd014c057347733a22ebc4d0.slice/crio-41ceb250e33f260550b642453727ead2255ae8c002fdf6e5508e99b715911fc6 WatchSource:0}: Error finding container 41ceb250e33f260550b642453727ead2255ae8c002fdf6e5508e99b715911fc6: Status 404 returned error can't find the container with id 41ceb250e33f260550b642453727ead2255ae8c002fdf6e5508e99b715911fc6 Jul 03 20:32:14.894246 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.894228 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 03 20:32:14.896510 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.896491 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 20:32:14.915975 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.915911 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bx9tt" Jul 03 20:32:14.927102 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.927081 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bx9tt" Jul 03 20:32:14.983145 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.983086 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" event={"ID":"6deb43b1e3d0667ae47de4e2350bbbcb","Type":"ContainerStarted","Data":"354180676d9357588e14748a202455251d72e0a42e2b9e725dfe53db839b3f09"} Jul 03 20:32:14.984209 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:14.984189 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" event={"ID":"99a5d64bbd014c057347733a22ebc4d0","Type":"ContainerStarted","Data":"41ceb250e33f260550b642453727ead2255ae8c002fdf6e5508e99b715911fc6"} Jul 03 20:32:15.671663 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.671624 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 20:32:15.849097 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.849044 2582 apiserver.go:52] "Watching apiserver" Jul 03 20:32:15.857413 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.857383 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 03 20:32:15.857864 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.857837 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-ng2rm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm","openshift-image-registry/node-ca-bs4f9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal","openshift-multus/multus-additional-cni-plugins-7jd92","openshift-multus/multus-cztbp","openshift-network-operator/iptables-alerter-th2hs","kube-system/global-pull-secret-syncer-chq7m","kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal","openshift-cluster-node-tuning-operator/tuned-8hfsx","openshift-dns/node-resolver-4lfd9","openshift-multus/network-metrics-daemon-s5vq9","openshift-network-diagnostics/network-check-target-vwpt4","openshift-ovn-kubernetes/ovnkube-node-j8gc9"] Jul 03 20:32:15.859592 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.859563 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.860915 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.860890 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:15.861013 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:15.860973 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:15.862159 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.862132 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bs4f9" Jul 03 20:32:15.862673 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.862648 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-798cm\"" Jul 03 20:32:15.862779 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.862706 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 03 20:32:15.863922 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.863898 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.865217 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.865199 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.866002 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.865982 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 03 20:32:15.866092 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.866007 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 03 20:32:15.870239 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.870202 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 03 20:32:15.870569 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.870551 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 03 20:32:15.870910 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.870893 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zpkt9\"" Jul 03 20:32:15.871161 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.871146 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 03 20:32:15.871363 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.871347 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 03 20:32:15.871589 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.871574 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-68f6l\"" Jul 03 20:32:15.872021 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.871990 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 03 20:32:15.872340 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.872321 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 03 20:32:15.872570 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.872555 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 03 20:32:15.873427 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.872931 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 03 20:32:15.873957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.873937 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 03 20:32:15.874171 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.874154 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-swpw8\"" Jul 03 20:32:15.874663 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.874642 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 03 20:32:15.874959 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.874942 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 03 20:32:15.875061 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.875043 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 03 20:32:15.875318 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.875300 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 03 20:32:15.875885 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.875856 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.876855 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.876746 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:15.876855 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:15.876843 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:15.878061 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.878022 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-th2hs" Jul 03 20:32:15.879705 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.879683 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:15.881157 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881136 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ng2rm" Jul 03 20:32:15.881264 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881208 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-cni-bin\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.881264 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881242 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8c1736d-e1e0-4fed-979c-4039a3b4c087-ovnkube-config\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.881375 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881268 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6a8fc1e-c013-45e0-a570-97d2e74f7feb-host\") pod \"node-ca-bs4f9\" (UID: \"f6a8fc1e-c013-45e0-a570-97d2e74f7feb\") " pod="openshift-image-registry/node-ca-bs4f9" Jul 03 20:32:15.881375 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881311 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5609fd01-5426-4ff6-97f5-a3d3367e85f0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.881375 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881338 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-var-lib-kubelet\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.881375 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881362 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-dbus\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:15.881554 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881393 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5609fd01-5426-4ff6-97f5-a3d3367e85f0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.881554 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881420 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4483b180-eb1d-488d-a652-4f12c1e41407-cni-binary-copy\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.881554 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881439 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-kubelet-config\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:15.881554 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881454 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-slash\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.881554 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881492 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5609fd01-5426-4ff6-97f5-a3d3367e85f0-cnibin\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.881554 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881516 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpx7t\" (UniqueName: \"kubernetes.io/projected/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-kube-api-access-cpx7t\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.881554 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881545 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-multus-conf-dir\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.881957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881586 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-etc-kubernetes\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.881957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881637 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-cni-netd\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.881957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881665 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-var-lib-kubelet\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.881957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881691 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5hzv\" (UniqueName: \"kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv\") pod \"network-check-target-vwpt4\" (UID: \"2494ce49-fb0d-4796-9b59-63dfd2d2c7fd\") " pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:15.881957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881721 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-var-lib-cni-multus\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.881957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881745 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-systemd-units\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.881957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881786 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-run-systemd\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.881957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881810 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-sys\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.881957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881835 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhcwv\" (UniqueName: \"kubernetes.io/projected/f6a8fc1e-c013-45e0-a570-97d2e74f7feb-kube-api-access-nhcwv\") pod \"node-ca-bs4f9\" (UID: \"f6a8fc1e-c013-45e0-a570-97d2e74f7feb\") " pod="openshift-image-registry/node-ca-bs4f9" Jul 03 20:32:15.881957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881859 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-multus-cni-dir\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.881957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881885 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-run-multus-certs\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.881957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881910 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxwrd\" (UniqueName: \"kubernetes.io/projected/4483b180-eb1d-488d-a652-4f12c1e41407-kube-api-access-nxwrd\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.881957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881936 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.881957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881962 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-kubernetes\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.881999 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-sysctl-conf\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882031 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-tuned\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882059 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-kubelet\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882097 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8c1736d-e1e0-4fed-979c-4039a3b4c087-env-overrides\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882123 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-systemd\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882146 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-run\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882169 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-lib-modules\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882200 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-run-netns\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882225 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-run-ovn\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882251 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-sysctl-d\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882278 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-tmp\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882306 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5609fd01-5426-4ff6-97f5-a3d3367e85f0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882332 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-system-cni-dir\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882358 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-run-k8s-cni-cncf-io\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882384 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-etc-openvswitch\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882407 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8c1736d-e1e0-4fed-979c-4039a3b4c087-ovn-node-metrics-cert\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.882552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882430 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6b87\" (UniqueName: \"kubernetes.io/projected/f8c1736d-e1e0-4fed-979c-4039a3b4c087-kube-api-access-v6b87\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882480 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-modprobe-d\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882531 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-host\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882568 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-node-log\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882595 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8c1736d-e1e0-4fed-979c-4039a3b4c087-ovnkube-script-lib\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882620 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-cnibin\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882682 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5609fd01-5426-4ff6-97f5-a3d3367e85f0-system-cni-dir\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882720 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5609fd01-5426-4ff6-97f5-a3d3367e85f0-os-release\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882744 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-os-release\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882786 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gmkq\" (UniqueName: \"kubernetes.io/projected/5609fd01-5426-4ff6-97f5-a3d3367e85f0-kube-api-access-2gmkq\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882812 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-run-netns\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882865 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882912 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-log-socket\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882943 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-sysconfig\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.882973 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5609fd01-5426-4ff6-97f5-a3d3367e85f0-cni-binary-copy\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.883001 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-var-lib-cni-bin\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.883279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.883026 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-hostroot\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.884054 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.883052 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4483b180-eb1d-488d-a652-4f12c1e41407-multus-daemon-config\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.884054 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.883078 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-var-lib-openvswitch\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.884054 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.883102 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f6a8fc1e-c013-45e0-a570-97d2e74f7feb-serviceca\") pod \"node-ca-bs4f9\" (UID: \"f6a8fc1e-c013-45e0-a570-97d2e74f7feb\") " pod="openshift-image-registry/node-ca-bs4f9" Jul 03 20:32:15.884054 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.883125 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-multus-socket-dir-parent\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.884054 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.883149 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-run-openvswitch\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.884054 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.883174 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-run-ovn-kubernetes\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.884054 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.883352 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4lfd9" Jul 03 20:32:15.884376 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.884183 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 03 20:32:15.884376 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.884264 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 03 20:32:15.884376 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.884311 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 03 20:32:15.884470 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.884263 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 03 20:32:15.884470 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.884458 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 03 20:32:15.884660 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.884642 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 03 20:32:15.884759 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.884673 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 03 20:32:15.885020 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.884962 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 03 20:32:15.885020 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.884984 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nbw95\"" Jul 03 20:32:15.885170 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.885073 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j7nt4\"" Jul 03 20:32:15.885170 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.885081 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:15.885170 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:15.885147 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:15.885170 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.885162 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 03 20:32:15.885451 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.885177 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fxhqj\"" Jul 03 20:32:15.885451 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.885279 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-njhrx\"" Jul 03 20:32:15.886036 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.886020 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g9k56\"" Jul 03 20:32:15.886285 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.886269 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 03 20:32:15.886644 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.886626 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 03 20:32:15.930453 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.930420 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 20:27:15 +0000 UTC" deadline="2027-12-19 04:22:50.17307731 +0000 UTC" Jul 03 20:32:15.930569 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.930455 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12799h50m34.242626831s" Jul 03 20:32:15.970967 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.970932 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 03 20:32:15.983332 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983307 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-socket-dir\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:15.983494 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983338 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e148c3c4-214b-4ed2-acea-69d3a0fe10b2-hosts-file\") pod \"node-resolver-4lfd9\" (UID: \"e148c3c4-214b-4ed2-acea-69d3a0fe10b2\") " pod="openshift-dns/node-resolver-4lfd9" Jul 03 20:32:15.983494 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983367 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs\") pod \"network-metrics-daemon-s5vq9\" (UID: \"499e1abf-db52-4ca2-9ea6-208a405209f3\") " pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:15.983494 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983391 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-kubernetes\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.983494 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983417 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-sysctl-conf\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.983494 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983439 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-tuned\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.983494 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983488 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3cba7612-a358-44b3-bce9-c3c87215cbcb-host-slash\") pod \"iptables-alerter-th2hs\" (UID: \"3cba7612-a358-44b3-bce9-c3c87215cbcb\") " pod="openshift-network-operator/iptables-alerter-th2hs" Jul 03 20:32:15.983800 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983510 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee651638-933f-4b70-a0b4-08438004b94d-agent-certs\") pod \"konnectivity-agent-ng2rm\" (UID: \"ee651638-933f-4b70-a0b4-08438004b94d\") " pod="kube-system/konnectivity-agent-ng2rm" Jul 03 20:32:15.983800 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983531 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-kubelet\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.983800 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983540 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-kubernetes\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.983800 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983555 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8c1736d-e1e0-4fed-979c-4039a3b4c087-env-overrides\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.983800 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983592 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-systemd\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.983800 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983620 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-sysctl-conf\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.983800 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983626 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-kubelet\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.983800 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983660 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-run\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.983800 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983676 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-lib-modules\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.983800 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.983701 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-etc-selinux\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:15.984307 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984149 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st7v9\" (UniqueName: \"kubernetes.io/projected/e148c3c4-214b-4ed2-acea-69d3a0fe10b2-kube-api-access-st7v9\") pod \"node-resolver-4lfd9\" (UID: \"e148c3c4-214b-4ed2-acea-69d3a0fe10b2\") " pod="openshift-dns/node-resolver-4lfd9" Jul 03 20:32:15.984307 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984207 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3cba7612-a358-44b3-bce9-c3c87215cbcb-iptables-alerter-script\") pod \"iptables-alerter-th2hs\" (UID: \"3cba7612-a358-44b3-bce9-c3c87215cbcb\") " pod="openshift-network-operator/iptables-alerter-th2hs" Jul 03 20:32:15.984307 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984228 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 03 20:32:15.984307 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984282 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-run-netns\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.984498 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984319 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-run-ovn\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.984498 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984351 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-sysctl-d\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.984498 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984373 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-tmp\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.984498 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984400 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee651638-933f-4b70-a0b4-08438004b94d-konnectivity-ca\") pod \"konnectivity-agent-ng2rm\" (UID: \"ee651638-933f-4b70-a0b4-08438004b94d\") " pod="kube-system/konnectivity-agent-ng2rm" Jul 03 20:32:15.984498 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984462 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5609fd01-5426-4ff6-97f5-a3d3367e85f0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.984735 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984502 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-system-cni-dir\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.984735 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984530 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-run-k8s-cni-cncf-io\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.984735 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984566 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-etc-openvswitch\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.984735 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984586 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-lib-modules\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.984735 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984598 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8c1736d-e1e0-4fed-979c-4039a3b4c087-ovn-node-metrics-cert\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.984735 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984592 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-run\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.984735 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984622 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6b87\" (UniqueName: \"kubernetes.io/projected/f8c1736d-e1e0-4fed-979c-4039a3b4c087-kube-api-access-v6b87\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.984735 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984645 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-modprobe-d\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.984735 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984672 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-host\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.984735 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984707 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-node-log\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984745 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8c1736d-e1e0-4fed-979c-4039a3b4c087-ovnkube-script-lib\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984780 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-registration-dir\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984806 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-sys-fs\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984805 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-systemd\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984845 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6jd6\" (UniqueName: \"kubernetes.io/projected/3cba7612-a358-44b3-bce9-c3c87215cbcb-kube-api-access-k6jd6\") pod \"iptables-alerter-th2hs\" (UID: \"3cba7612-a358-44b3-bce9-c3c87215cbcb\") " pod="openshift-network-operator/iptables-alerter-th2hs" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984875 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-cnibin\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984874 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-run-k8s-cni-cncf-io\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984952 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-system-cni-dir\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.984974 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-run-netns\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.985002 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-node-log\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.985028 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8c1736d-e1e0-4fed-979c-4039a3b4c087-env-overrides\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.985079 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-etc-openvswitch\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.985033 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-run-ovn\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.985151 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-sysctl-d\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.985230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.985228 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-modprobe-d\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.985959 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.985480 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-host\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.985959 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.985631 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-cnibin\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.985959 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.985811 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5609fd01-5426-4ff6-97f5-a3d3367e85f0-system-cni-dir\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.985959 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.985864 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5609fd01-5426-4ff6-97f5-a3d3367e85f0-os-release\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.985959 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.985903 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-device-dir\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:15.985959 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.985936 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-os-release\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.986197 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.985963 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2b89\" (UniqueName: \"kubernetes.io/projected/499e1abf-db52-4ca2-9ea6-208a405209f3-kube-api-access-b2b89\") pod \"network-metrics-daemon-s5vq9\" (UID: \"499e1abf-db52-4ca2-9ea6-208a405209f3\") " pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:15.986197 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.985994 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gmkq\" (UniqueName: \"kubernetes.io/projected/5609fd01-5426-4ff6-97f5-a3d3367e85f0-kube-api-access-2gmkq\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.986197 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986021 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-run-netns\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.986197 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986049 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:15.986197 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986061 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8c1736d-e1e0-4fed-979c-4039a3b4c087-ovnkube-script-lib\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.986197 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986077 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-log-socket\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.986197 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986119 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-log-socket\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.986197 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986127 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-sysconfig\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.986197 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986162 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5609fd01-5426-4ff6-97f5-a3d3367e85f0-cni-binary-copy\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.986197 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986186 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-var-lib-cni-bin\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.986197 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986190 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-sysconfig\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.986624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986214 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-hostroot\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.986624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986246 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4483b180-eb1d-488d-a652-4f12c1e41407-multus-daemon-config\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.986624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986251 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5609fd01-5426-4ff6-97f5-a3d3367e85f0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.986624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986290 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-os-release\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.986624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986366 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-hostroot\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.986624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986398 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-var-lib-openvswitch\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.986624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986427 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f6a8fc1e-c013-45e0-a570-97d2e74f7feb-serviceca\") pod \"node-ca-bs4f9\" (UID: \"f6a8fc1e-c013-45e0-a570-97d2e74f7feb\") " pod="openshift-image-registry/node-ca-bs4f9" Jul 03 20:32:15.986624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986448 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-multus-socket-dir-parent\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.986624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986475 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-run-openvswitch\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.986624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986501 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-run-ovn-kubernetes\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.986624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986529 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-cni-bin\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.986624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986569 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8c1736d-e1e0-4fed-979c-4039a3b4c087-ovnkube-config\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.986624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986599 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6a8fc1e-c013-45e0-a570-97d2e74f7feb-host\") pod \"node-ca-bs4f9\" (UID: \"f6a8fc1e-c013-45e0-a570-97d2e74f7feb\") " pod="openshift-image-registry/node-ca-bs4f9" Jul 03 20:32:15.986624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986625 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5609fd01-5426-4ff6-97f5-a3d3367e85f0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.987229 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986655 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-var-lib-kubelet\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.987229 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986684 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-dbus\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:15.987229 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986707 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5609fd01-5426-4ff6-97f5-a3d3367e85f0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.987229 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986876 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4483b180-eb1d-488d-a652-4f12c1e41407-cni-binary-copy\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.987229 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986906 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-kubelet-config\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:15.987229 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986932 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-slash\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.987229 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986958 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5609fd01-5426-4ff6-97f5-a3d3367e85f0-cnibin\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.987229 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986981 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpx7t\" (UniqueName: \"kubernetes.io/projected/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-kube-api-access-cpx7t\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.987229 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987009 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-multus-conf-dir\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.987229 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987055 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-run-netns\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.987229 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987075 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-var-lib-cni-bin\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.987229 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987107 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-etc-kubernetes\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.987229 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987165 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6a8fc1e-c013-45e0-a570-97d2e74f7feb-host\") pod \"node-ca-bs4f9\" (UID: \"f6a8fc1e-c013-45e0-a570-97d2e74f7feb\") " pod="openshift-image-registry/node-ca-bs4f9" Jul 03 20:32:15.987229 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:15.987178 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 20:32:15.987834 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987237 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5609fd01-5426-4ff6-97f5-a3d3367e85f0-os-release\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.987834 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:15.987256 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret podName:50e502c5-dd61-4c6f-9bcf-6de839bf5fbb nodeName:}" failed. No retries permitted until 2026-07-03 20:32:16.487233273 +0000 UTC m=+2.945505711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret") pod "global-pull-secret-syncer-chq7m" (UID: "50e502c5-dd61-4c6f-9bcf-6de839bf5fbb") : object "kube-system"/"original-pull-secret" not registered Jul 03 20:32:15.987834 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987274 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-etc-kubernetes\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.987834 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987357 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-var-lib-kubelet\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.987834 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987451 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5609fd01-5426-4ff6-97f5-a3d3367e85f0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.987834 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987539 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-var-lib-openvswitch\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.987834 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987542 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-dbus\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:15.987834 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987631 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-cni-netd\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.987834 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987670 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-var-lib-kubelet\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.987834 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987704 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5hzv\" (UniqueName: \"kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv\") pod \"network-check-target-vwpt4\" (UID: \"2494ce49-fb0d-4796-9b59-63dfd2d2c7fd\") " pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:15.987834 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987751 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e148c3c4-214b-4ed2-acea-69d3a0fe10b2-tmp-dir\") pod \"node-resolver-4lfd9\" (UID: \"e148c3c4-214b-4ed2-acea-69d3a0fe10b2\") " pod="openshift-dns/node-resolver-4lfd9" Jul 03 20:32:15.987834 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987786 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-var-lib-cni-multus\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.988345 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987897 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-systemd-units\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.988345 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987935 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-run-systemd\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.988345 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987969 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-sys\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.988345 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.987996 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhcwv\" (UniqueName: \"kubernetes.io/projected/f6a8fc1e-c013-45e0-a570-97d2e74f7feb-kube-api-access-nhcwv\") pod \"node-ca-bs4f9\" (UID: \"f6a8fc1e-c013-45e0-a570-97d2e74f7feb\") " pod="openshift-image-registry/node-ca-bs4f9" Jul 03 20:32:15.988345 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988031 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:15.988345 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988067 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvpbv\" (UniqueName: \"kubernetes.io/projected/e46049a0-7317-495b-8e8d-7663ac8fd178-kube-api-access-fvpbv\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:15.988345 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988079 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f6a8fc1e-c013-45e0-a570-97d2e74f7feb-serviceca\") pod \"node-ca-bs4f9\" (UID: \"f6a8fc1e-c013-45e0-a570-97d2e74f7feb\") " pod="openshift-image-registry/node-ca-bs4f9" Jul 03 20:32:15.988345 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988139 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5609fd01-5426-4ff6-97f5-a3d3367e85f0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.988345 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988280 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-multus-socket-dir-parent\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.988345 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988279 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-multus-cni-dir\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.988345 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988329 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-run-multus-certs\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.988791 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988367 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxwrd\" (UniqueName: \"kubernetes.io/projected/4483b180-eb1d-488d-a652-4f12c1e41407-kube-api-access-nxwrd\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.988791 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988383 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-cni-netd\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.988791 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988404 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.988791 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988454 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-var-lib-kubelet\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.988791 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988744 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4483b180-eb1d-488d-a652-4f12c1e41407-cni-binary-copy\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.988791 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988760 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-var-lib-cni-multus\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.988791 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.986875 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5609fd01-5426-4ff6-97f5-a3d3367e85f0-cni-binary-copy\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.989100 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988814 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-kubelet-config\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:15.989100 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988866 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-systemd-units\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.989100 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988921 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-slash\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.989100 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.988944 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-run-systemd\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.989100 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.989000 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5609fd01-5426-4ff6-97f5-a3d3367e85f0-cnibin\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.989100 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.989007 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-sys\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.989335 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.989125 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-run-openvswitch\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.989335 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.989147 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-multus-conf-dir\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.989335 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.989215 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-run-ovn-kubernetes\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.989335 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.989282 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-multus-cni-dir\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.989498 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.989382 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5609fd01-5426-4ff6-97f5-a3d3367e85f0-system-cni-dir\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:15.989498 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.989416 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-cni-bin\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.989943 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.989919 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8c1736d-e1e0-4fed-979c-4039a3b4c087-ovnkube-config\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.992415 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.990250 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-etc-tuned\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.992415 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.989956 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4483b180-eb1d-488d-a652-4f12c1e41407-multus-daemon-config\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.992415 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.990092 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4483b180-eb1d-488d-a652-4f12c1e41407-host-run-multus-certs\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:15.992415 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.990072 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8c1736d-e1e0-4fed-979c-4039a3b4c087-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.992415 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.991925 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8c1736d-e1e0-4fed-979c-4039a3b4c087-ovn-node-metrics-cert\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:15.993954 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.993928 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-tmp\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:15.996181 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:15.996161 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6b87\" (UniqueName: \"kubernetes.io/projected/f8c1736d-e1e0-4fed-979c-4039a3b4c087-kube-api-access-v6b87\") pod \"ovnkube-node-j8gc9\" (UID: \"f8c1736d-e1e0-4fed-979c-4039a3b4c087\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:16.001191 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.001167 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 20:32:16.001299 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.001224 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 20:32:16.001299 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.001239 2582 projected.go:196] Error preparing data for projected volume kube-api-access-v5hzv for pod openshift-network-diagnostics/network-check-target-vwpt4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:32:16.001452 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.001310 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv podName:2494ce49-fb0d-4796-9b59-63dfd2d2c7fd nodeName:}" failed. No retries permitted until 2026-07-03 20:32:16.501288652 +0000 UTC m=+2.959561171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v5hzv" (UniqueName: "kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv") pod "network-check-target-vwpt4" (UID: "2494ce49-fb0d-4796-9b59-63dfd2d2c7fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:32:16.001985 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.001962 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gmkq\" (UniqueName: \"kubernetes.io/projected/5609fd01-5426-4ff6-97f5-a3d3367e85f0-kube-api-access-2gmkq\") pod \"multus-additional-cni-plugins-7jd92\" (UID: \"5609fd01-5426-4ff6-97f5-a3d3367e85f0\") " pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:16.003583 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.003566 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxwrd\" (UniqueName: \"kubernetes.io/projected/4483b180-eb1d-488d-a652-4f12c1e41407-kube-api-access-nxwrd\") pod \"multus-cztbp\" (UID: \"4483b180-eb1d-488d-a652-4f12c1e41407\") " pod="openshift-multus/multus-cztbp" Jul 03 20:32:16.003928 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.003901 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpx7t\" (UniqueName: \"kubernetes.io/projected/cfe616fb-4d9c-4ece-ba6a-c5c58e496738-kube-api-access-cpx7t\") pod \"tuned-8hfsx\" (UID: \"cfe616fb-4d9c-4ece-ba6a-c5c58e496738\") " pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:16.004156 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.004138 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhcwv\" (UniqueName: \"kubernetes.io/projected/f6a8fc1e-c013-45e0-a570-97d2e74f7feb-kube-api-access-nhcwv\") pod \"node-ca-bs4f9\" (UID: \"f6a8fc1e-c013-45e0-a570-97d2e74f7feb\") " pod="openshift-image-registry/node-ca-bs4f9" Jul 03 20:32:16.089574 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.089528 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-registration-dir\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.089760 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.089587 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-sys-fs\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.089760 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.089627 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6jd6\" (UniqueName: \"kubernetes.io/projected/3cba7612-a358-44b3-bce9-c3c87215cbcb-kube-api-access-k6jd6\") pod \"iptables-alerter-th2hs\" (UID: \"3cba7612-a358-44b3-bce9-c3c87215cbcb\") " pod="openshift-network-operator/iptables-alerter-th2hs" Jul 03 20:32:16.089760 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.089659 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-device-dir\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.089760 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.089681 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-registration-dir\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.089760 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.089691 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2b89\" (UniqueName: \"kubernetes.io/projected/499e1abf-db52-4ca2-9ea6-208a405209f3-kube-api-access-b2b89\") pod \"network-metrics-daemon-s5vq9\" (UID: \"499e1abf-db52-4ca2-9ea6-208a405209f3\") " pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:16.090066 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.089748 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-sys-fs\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.090066 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.089852 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e148c3c4-214b-4ed2-acea-69d3a0fe10b2-tmp-dir\") pod \"node-resolver-4lfd9\" (UID: \"e148c3c4-214b-4ed2-acea-69d3a0fe10b2\") " pod="openshift-dns/node-resolver-4lfd9" Jul 03 20:32:16.090066 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.089891 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.090066 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.089939 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvpbv\" (UniqueName: \"kubernetes.io/projected/e46049a0-7317-495b-8e8d-7663ac8fd178-kube-api-access-fvpbv\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.090066 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.089964 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-socket-dir\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.090066 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.089978 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-device-dir\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.090066 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.089982 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e148c3c4-214b-4ed2-acea-69d3a0fe10b2-hosts-file\") pod \"node-resolver-4lfd9\" (UID: \"e148c3c4-214b-4ed2-acea-69d3a0fe10b2\") " pod="openshift-dns/node-resolver-4lfd9" Jul 03 20:32:16.090066 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.090026 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs\") pod \"network-metrics-daemon-s5vq9\" (UID: \"499e1abf-db52-4ca2-9ea6-208a405209f3\") " pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:16.090066 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.090043 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e148c3c4-214b-4ed2-acea-69d3a0fe10b2-hosts-file\") pod \"node-resolver-4lfd9\" (UID: \"e148c3c4-214b-4ed2-acea-69d3a0fe10b2\") " pod="openshift-dns/node-resolver-4lfd9" Jul 03 20:32:16.090066 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.090056 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3cba7612-a358-44b3-bce9-c3c87215cbcb-host-slash\") pod \"iptables-alerter-th2hs\" (UID: \"3cba7612-a358-44b3-bce9-c3c87215cbcb\") " pod="openshift-network-operator/iptables-alerter-th2hs" Jul 03 20:32:16.090388 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.090082 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee651638-933f-4b70-a0b4-08438004b94d-agent-certs\") pod \"konnectivity-agent-ng2rm\" (UID: \"ee651638-933f-4b70-a0b4-08438004b94d\") " pod="kube-system/konnectivity-agent-ng2rm" Jul 03 20:32:16.090388 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.090116 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-etc-selinux\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.090388 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.090144 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st7v9\" (UniqueName: \"kubernetes.io/projected/e148c3c4-214b-4ed2-acea-69d3a0fe10b2-kube-api-access-st7v9\") pod \"node-resolver-4lfd9\" (UID: \"e148c3c4-214b-4ed2-acea-69d3a0fe10b2\") " pod="openshift-dns/node-resolver-4lfd9" Jul 03 20:32:16.090388 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.090171 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3cba7612-a358-44b3-bce9-c3c87215cbcb-iptables-alerter-script\") pod \"iptables-alerter-th2hs\" (UID: \"3cba7612-a358-44b3-bce9-c3c87215cbcb\") " pod="openshift-network-operator/iptables-alerter-th2hs" Jul 03 20:32:16.090388 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.090204 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee651638-933f-4b70-a0b4-08438004b94d-konnectivity-ca\") pod \"konnectivity-agent-ng2rm\" (UID: \"ee651638-933f-4b70-a0b4-08438004b94d\") " pod="kube-system/konnectivity-agent-ng2rm" Jul 03 20:32:16.090388 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.090261 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e148c3c4-214b-4ed2-acea-69d3a0fe10b2-tmp-dir\") pod \"node-resolver-4lfd9\" (UID: \"e148c3c4-214b-4ed2-acea-69d3a0fe10b2\") " pod="openshift-dns/node-resolver-4lfd9" Jul 03 20:32:16.090388 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.090340 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-socket-dir\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.090687 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.090426 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.090687 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.090459 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e46049a0-7317-495b-8e8d-7663ac8fd178-etc-selinux\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.090687 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.090485 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3cba7612-a358-44b3-bce9-c3c87215cbcb-host-slash\") pod \"iptables-alerter-th2hs\" (UID: \"3cba7612-a358-44b3-bce9-c3c87215cbcb\") " pod="openshift-network-operator/iptables-alerter-th2hs" Jul 03 20:32:16.090687 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.090144 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:32:16.090687 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.090570 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs podName:499e1abf-db52-4ca2-9ea6-208a405209f3 nodeName:}" failed. No retries permitted until 2026-07-03 20:32:16.590541554 +0000 UTC m=+3.048813977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs") pod "network-metrics-daemon-s5vq9" (UID: "499e1abf-db52-4ca2-9ea6-208a405209f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:32:16.090967 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.090878 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ee651638-933f-4b70-a0b4-08438004b94d-konnectivity-ca\") pod \"konnectivity-agent-ng2rm\" (UID: \"ee651638-933f-4b70-a0b4-08438004b94d\") " pod="kube-system/konnectivity-agent-ng2rm" Jul 03 20:32:16.091230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.091207 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3cba7612-a358-44b3-bce9-c3c87215cbcb-iptables-alerter-script\") pod \"iptables-alerter-th2hs\" (UID: \"3cba7612-a358-44b3-bce9-c3c87215cbcb\") " pod="openshift-network-operator/iptables-alerter-th2hs" Jul 03 20:32:16.092935 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.092915 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ee651638-933f-4b70-a0b4-08438004b94d-agent-certs\") pod \"konnectivity-agent-ng2rm\" (UID: \"ee651638-933f-4b70-a0b4-08438004b94d\") " pod="kube-system/konnectivity-agent-ng2rm" Jul 03 20:32:16.107022 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.106999 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvpbv\" (UniqueName: \"kubernetes.io/projected/e46049a0-7317-495b-8e8d-7663ac8fd178-kube-api-access-fvpbv\") pod \"aws-ebs-csi-driver-node-sn9hm\" (UID: \"e46049a0-7317-495b-8e8d-7663ac8fd178\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.107148 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.107130 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6jd6\" (UniqueName: \"kubernetes.io/projected/3cba7612-a358-44b3-bce9-c3c87215cbcb-kube-api-access-k6jd6\") pod \"iptables-alerter-th2hs\" (UID: \"3cba7612-a358-44b3-bce9-c3c87215cbcb\") " pod="openshift-network-operator/iptables-alerter-th2hs" Jul 03 20:32:16.107382 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.107363 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-st7v9\" (UniqueName: \"kubernetes.io/projected/e148c3c4-214b-4ed2-acea-69d3a0fe10b2-kube-api-access-st7v9\") pod \"node-resolver-4lfd9\" (UID: \"e148c3c4-214b-4ed2-acea-69d3a0fe10b2\") " pod="openshift-dns/node-resolver-4lfd9" Jul 03 20:32:16.107693 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.107676 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2b89\" (UniqueName: \"kubernetes.io/projected/499e1abf-db52-4ca2-9ea6-208a405209f3-kube-api-access-b2b89\") pod \"network-metrics-daemon-s5vq9\" (UID: \"499e1abf-db52-4ca2-9ea6-208a405209f3\") " pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:16.180070 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.179982 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" Jul 03 20:32:16.187354 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.187328 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bs4f9" Jul 03 20:32:16.196579 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.196557 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:16.202597 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.202573 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7jd92" Jul 03 20:32:16.209511 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.209490 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cztbp" Jul 03 20:32:16.217344 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.217326 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-th2hs" Jul 03 20:32:16.225034 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.225008 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" Jul 03 20:32:16.231860 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.231836 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ng2rm" Jul 03 20:32:16.237557 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.237538 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4lfd9" Jul 03 20:32:16.358626 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.358596 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 20:32:16.494026 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.493941 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:16.494193 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.494086 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 20:32:16.494193 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.494165 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret podName:50e502c5-dd61-4c6f-9bcf-6de839bf5fbb nodeName:}" failed. No retries permitted until 2026-07-03 20:32:17.494143447 +0000 UTC m=+3.952415865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret") pod "global-pull-secret-syncer-chq7m" (UID: "50e502c5-dd61-4c6f-9bcf-6de839bf5fbb") : object "kube-system"/"original-pull-secret" not registered Jul 03 20:32:16.594686 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.594642 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5hzv\" (UniqueName: \"kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv\") pod \"network-check-target-vwpt4\" (UID: \"2494ce49-fb0d-4796-9b59-63dfd2d2c7fd\") " pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:16.594887 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.594709 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs\") pod \"network-metrics-daemon-s5vq9\" (UID: \"499e1abf-db52-4ca2-9ea6-208a405209f3\") " pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:16.594887 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.594821 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 20:32:16.594887 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.594850 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 20:32:16.594887 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.594862 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:32:16.595041 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.594934 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs podName:499e1abf-db52-4ca2-9ea6-208a405209f3 nodeName:}" failed. No retries permitted until 2026-07-03 20:32:17.594917033 +0000 UTC m=+4.053189451 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs") pod "network-metrics-daemon-s5vq9" (UID: "499e1abf-db52-4ca2-9ea6-208a405209f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:32:16.595041 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.594864 2582 projected.go:196] Error preparing data for projected volume kube-api-access-v5hzv for pod openshift-network-diagnostics/network-check-target-vwpt4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:32:16.595041 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.594976 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv podName:2494ce49-fb0d-4796-9b59-63dfd2d2c7fd nodeName:}" failed. No retries permitted until 2026-07-03 20:32:17.594968773 +0000 UTC m=+4.053241191 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-v5hzv" (UniqueName: "kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv") pod "network-check-target-vwpt4" (UID: "2494ce49-fb0d-4796-9b59-63dfd2d2c7fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:32:16.838300 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:16.838261 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6a8fc1e_c013_45e0_a570_97d2e74f7feb.slice/crio-905cffc7933026bf3a0598368a77a49aa6e7b464acc1af79abe0c49bd82d73ca WatchSource:0}: Error finding container 905cffc7933026bf3a0598368a77a49aa6e7b464acc1af79abe0c49bd82d73ca: Status 404 returned error can't find the container with id 905cffc7933026bf3a0598368a77a49aa6e7b464acc1af79abe0c49bd82d73ca Jul 03 20:32:16.839775 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:16.839721 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode46049a0_7317_495b_8e8d_7663ac8fd178.slice/crio-88c1a5b5f9ec7fcace14fa9fd89894dce4ed1be438511af6eadd8c372b617c59 WatchSource:0}: Error finding container 88c1a5b5f9ec7fcace14fa9fd89894dce4ed1be438511af6eadd8c372b617c59: Status 404 returned error can't find the container with id 88c1a5b5f9ec7fcace14fa9fd89894dce4ed1be438511af6eadd8c372b617c59 Jul 03 20:32:16.841550 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:16.841286 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4483b180_eb1d_488d_a652_4f12c1e41407.slice/crio-b513d57681525efbf1d68f888fa89d1d2ee6533069fbee6067a1d92a5e4886b8 WatchSource:0}: Error finding container b513d57681525efbf1d68f888fa89d1d2ee6533069fbee6067a1d92a5e4886b8: Status 404 returned error can't find the container with id b513d57681525efbf1d68f888fa89d1d2ee6533069fbee6067a1d92a5e4886b8 Jul 03 20:32:16.842606 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:16.842505 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c1736d_e1e0_4fed_979c_4039a3b4c087.slice/crio-6dc3396a85402cab969b7a2fc643f54dddc37e8311c52cfad2c4fbb88b1944f2 WatchSource:0}: Error finding container 6dc3396a85402cab969b7a2fc643f54dddc37e8311c52cfad2c4fbb88b1944f2: Status 404 returned error can't find the container with id 6dc3396a85402cab969b7a2fc643f54dddc37e8311c52cfad2c4fbb88b1944f2 Jul 03 20:32:16.931737 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.931548 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 20:27:15 +0000 UTC" deadline="2028-02-27 16:43:48.250041066 +0000 UTC" Jul 03 20:32:16.932162 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.931739 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14492h11m31.318307296s" Jul 03 20:32:16.980223 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.980191 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:16.980377 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:16.980317 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:16.998562 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:16.998522 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" event={"ID":"6deb43b1e3d0667ae47de4e2350bbbcb","Type":"ContainerStarted","Data":"cf8d596fea6aaca0d581b5e63b340f99d6784b98aab017b70a095838a5e842ee"} Jul 03 20:32:17.000307 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.000275 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4lfd9" event={"ID":"e148c3c4-214b-4ed2-acea-69d3a0fe10b2","Type":"ContainerStarted","Data":"191d8133c2a87d25ff78a59ac224ce87d4ac76fabfe000f00f708d872d0f757b"} Jul 03 20:32:17.001359 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.001331 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-th2hs" event={"ID":"3cba7612-a358-44b3-bce9-c3c87215cbcb","Type":"ContainerStarted","Data":"243581f10a0b468f7ed78fdf369b469a4e97c305ff272470e91cf3bcf11c98d9"} Jul 03 20:32:17.003385 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.003292 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" event={"ID":"f8c1736d-e1e0-4fed-979c-4039a3b4c087","Type":"ContainerStarted","Data":"6dc3396a85402cab969b7a2fc643f54dddc37e8311c52cfad2c4fbb88b1944f2"} Jul 03 20:32:17.004457 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.004437 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" event={"ID":"e46049a0-7317-495b-8e8d-7663ac8fd178","Type":"ContainerStarted","Data":"88c1a5b5f9ec7fcace14fa9fd89894dce4ed1be438511af6eadd8c372b617c59"} Jul 03 20:32:17.007503 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.007479 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jd92" event={"ID":"5609fd01-5426-4ff6-97f5-a3d3367e85f0","Type":"ContainerStarted","Data":"d6f032885198ca450d89d11560a8ddd9253a9d12462077ebd4d75b6c3c7a2929"} Jul 03 20:32:17.009902 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.009877 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" event={"ID":"cfe616fb-4d9c-4ece-ba6a-c5c58e496738","Type":"ContainerStarted","Data":"979c9912fce487432c6d09c288619bc16e7dc61c1563d52f917126398e1dba34"} Jul 03 20:32:17.011379 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.011356 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ng2rm" event={"ID":"ee651638-933f-4b70-a0b4-08438004b94d","Type":"ContainerStarted","Data":"5fd4110ffbb6de8b5632eb2c3d38f8d6d8457e85d6cbc4700ae63efe2e056d4f"} Jul 03 20:32:17.014117 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.014095 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cztbp" event={"ID":"4483b180-eb1d-488d-a652-4f12c1e41407","Type":"ContainerStarted","Data":"b513d57681525efbf1d68f888fa89d1d2ee6533069fbee6067a1d92a5e4886b8"} Jul 03 20:32:17.016197 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.016177 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bs4f9" event={"ID":"f6a8fc1e-c013-45e0-a570-97d2e74f7feb","Type":"ContainerStarted","Data":"905cffc7933026bf3a0598368a77a49aa6e7b464acc1af79abe0c49bd82d73ca"} Jul 03 20:32:17.504030 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.503990 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 20:32:17.508425 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.507842 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:17.508425 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:17.508047 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 20:32:17.508425 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:17.508110 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret podName:50e502c5-dd61-4c6f-9bcf-6de839bf5fbb nodeName:}" failed. No retries permitted until 2026-07-03 20:32:19.508091646 +0000 UTC m=+5.966364082 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret") pod "global-pull-secret-syncer-chq7m" (UID: "50e502c5-dd61-4c6f-9bcf-6de839bf5fbb") : object "kube-system"/"original-pull-secret" not registered Jul 03 20:32:17.609236 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.609187 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5hzv\" (UniqueName: \"kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv\") pod \"network-check-target-vwpt4\" (UID: \"2494ce49-fb0d-4796-9b59-63dfd2d2c7fd\") " pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:17.609432 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.609256 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs\") pod \"network-metrics-daemon-s5vq9\" (UID: \"499e1abf-db52-4ca2-9ea6-208a405209f3\") " pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:17.609495 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:17.609439 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:32:17.609563 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:17.609511 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs podName:499e1abf-db52-4ca2-9ea6-208a405209f3 nodeName:}" failed. No retries permitted until 2026-07-03 20:32:19.609490612 +0000 UTC m=+6.067763034 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs") pod "network-metrics-daemon-s5vq9" (UID: "499e1abf-db52-4ca2-9ea6-208a405209f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:32:17.610055 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:17.609969 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 20:32:17.610055 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:17.609995 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 20:32:17.610055 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:17.610007 2582 projected.go:196] Error preparing data for projected volume kube-api-access-v5hzv for pod openshift-network-diagnostics/network-check-target-vwpt4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:32:17.610055 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:17.610055 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv podName:2494ce49-fb0d-4796-9b59-63dfd2d2c7fd nodeName:}" failed. No retries permitted until 2026-07-03 20:32:19.610039593 +0000 UTC m=+6.068312032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-v5hzv" (UniqueName: "kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv") pod "network-check-target-vwpt4" (UID: "2494ce49-fb0d-4796-9b59-63dfd2d2c7fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:32:17.980000 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.979962 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:17.980530 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:17.980010 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:17.980530 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:17.980138 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:17.980530 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:17.980298 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:18.041272 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:18.040269 2582 generic.go:356] "Generic (PLEG): container finished" podID="99a5d64bbd014c057347733a22ebc4d0" containerID="381996d48cbc82f361b8218a537d4c04d1ddbf21ec8f7854c2aaa785b2897a1d" exitCode=0 Jul 03 20:32:18.041272 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:18.040988 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" event={"ID":"99a5d64bbd014c057347733a22ebc4d0","Type":"ContainerDied","Data":"381996d48cbc82f361b8218a537d4c04d1ddbf21ec8f7854c2aaa785b2897a1d"} Jul 03 20:32:18.064472 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:18.063352 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-68.ec2.internal" podStartSLOduration=3.063330098 podStartE2EDuration="3.063330098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:32:17.01510238 +0000 UTC m=+3.473374823" watchObservedRunningTime="2026-07-03 20:32:18.063330098 +0000 UTC m=+4.521602543" Jul 03 20:32:18.981039 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:18.980952 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:18.981584 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:18.981097 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:19.074125 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:19.074085 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" event={"ID":"99a5d64bbd014c057347733a22ebc4d0","Type":"ContainerStarted","Data":"2fe074fc89fd2b5211b6a8f9b801c579cf68106b5dc68201667ed59d2f8f07ad"} Jul 03 20:32:19.526539 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:19.526498 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:19.526724 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:19.526712 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 20:32:19.526848 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:19.526783 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret podName:50e502c5-dd61-4c6f-9bcf-6de839bf5fbb nodeName:}" failed. No retries permitted until 2026-07-03 20:32:23.526762172 +0000 UTC m=+9.985034613 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret") pod "global-pull-secret-syncer-chq7m" (UID: "50e502c5-dd61-4c6f-9bcf-6de839bf5fbb") : object "kube-system"/"original-pull-secret" not registered Jul 03 20:32:19.628004 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:19.627964 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5hzv\" (UniqueName: \"kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv\") pod \"network-check-target-vwpt4\" (UID: \"2494ce49-fb0d-4796-9b59-63dfd2d2c7fd\") " pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:19.628191 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:19.628025 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs\") pod \"network-metrics-daemon-s5vq9\" (UID: \"499e1abf-db52-4ca2-9ea6-208a405209f3\") " pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:19.628191 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:19.628174 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:32:19.628298 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:19.628252 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs podName:499e1abf-db52-4ca2-9ea6-208a405209f3 nodeName:}" failed. No retries permitted until 2026-07-03 20:32:23.628219853 +0000 UTC m=+10.086492274 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs") pod "network-metrics-daemon-s5vq9" (UID: "499e1abf-db52-4ca2-9ea6-208a405209f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:32:19.628705 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:19.628682 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 20:32:19.628705 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:19.628709 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 20:32:19.628908 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:19.628721 2582 projected.go:196] Error preparing data for projected volume kube-api-access-v5hzv for pod openshift-network-diagnostics/network-check-target-vwpt4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:32:19.628908 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:19.628782 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv podName:2494ce49-fb0d-4796-9b59-63dfd2d2c7fd nodeName:}" failed. No retries permitted until 2026-07-03 20:32:23.628752156 +0000 UTC m=+10.087024576 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-v5hzv" (UniqueName: "kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv") pod "network-check-target-vwpt4" (UID: "2494ce49-fb0d-4796-9b59-63dfd2d2c7fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:32:19.982798 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:19.982761 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:19.983343 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:19.982822 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:19.983343 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:19.982941 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:19.983343 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:19.983043 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:20.980665 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:20.980171 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:20.980665 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:20.980295 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:21.983095 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:21.982564 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:21.983095 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:21.982714 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:21.983095 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:21.982839 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:21.983095 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:21.982925 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:22.980801 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:22.980762 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:22.980973 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:22.980897 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:23.563302 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:23.563263 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:23.563740 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:23.563428 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 20:32:23.563740 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:23.563521 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret podName:50e502c5-dd61-4c6f-9bcf-6de839bf5fbb nodeName:}" failed. No retries permitted until 2026-07-03 20:32:31.563488957 +0000 UTC m=+18.021761379 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret") pod "global-pull-secret-syncer-chq7m" (UID: "50e502c5-dd61-4c6f-9bcf-6de839bf5fbb") : object "kube-system"/"original-pull-secret" not registered Jul 03 20:32:23.664237 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:23.664124 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5hzv\" (UniqueName: \"kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv\") pod \"network-check-target-vwpt4\" (UID: \"2494ce49-fb0d-4796-9b59-63dfd2d2c7fd\") " pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:23.664237 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:23.664173 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs\") pod \"network-metrics-daemon-s5vq9\" (UID: \"499e1abf-db52-4ca2-9ea6-208a405209f3\") " pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:23.664485 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:23.664303 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:32:23.664485 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:23.664309 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 20:32:23.664485 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:23.664329 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 20:32:23.664485 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:23.664339 2582 projected.go:196] Error preparing data for projected volume kube-api-access-v5hzv for pod openshift-network-diagnostics/network-check-target-vwpt4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:32:23.664485 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:23.664366 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs podName:499e1abf-db52-4ca2-9ea6-208a405209f3 nodeName:}" failed. No retries permitted until 2026-07-03 20:32:31.664346385 +0000 UTC m=+18.122618805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs") pod "network-metrics-daemon-s5vq9" (UID: "499e1abf-db52-4ca2-9ea6-208a405209f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:32:23.664485 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:23.664386 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv podName:2494ce49-fb0d-4796-9b59-63dfd2d2c7fd nodeName:}" failed. No retries permitted until 2026-07-03 20:32:31.664374576 +0000 UTC m=+18.122646997 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-v5hzv" (UniqueName: "kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv") pod "network-check-target-vwpt4" (UID: "2494ce49-fb0d-4796-9b59-63dfd2d2c7fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:32:23.982114 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:23.982083 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:23.982266 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:23.982191 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:23.982511 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:23.982465 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:23.982589 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:23.982529 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:24.980633 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:24.980173 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:24.980633 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:24.980303 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:25.981284 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:25.980822 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:25.981284 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:25.980836 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:25.981284 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:25.980959 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:25.981284 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:25.981058 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:26.980583 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:26.980547 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:26.980728 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:26.980653 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:27.980013 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:27.979991 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:27.980281 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:27.980028 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:27.980281 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:27.980120 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:27.980281 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:27.980199 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:28.980193 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:28.980154 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:28.980654 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:28.980281 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:29.980711 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:29.980679 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:29.981131 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:29.980816 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:29.981131 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:29.980859 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:29.981131 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:29.980969 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:30.980900 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:30.980867 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:30.981432 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:30.980995 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:31.625541 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:31.625484 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:31.625709 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:31.625618 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 20:32:31.625709 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:31.625687 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret podName:50e502c5-dd61-4c6f-9bcf-6de839bf5fbb nodeName:}" failed. No retries permitted until 2026-07-03 20:32:47.625666385 +0000 UTC m=+34.083938819 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret") pod "global-pull-secret-syncer-chq7m" (UID: "50e502c5-dd61-4c6f-9bcf-6de839bf5fbb") : object "kube-system"/"original-pull-secret" not registered Jul 03 20:32:31.726108 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:31.726073 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5hzv\" (UniqueName: \"kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv\") pod \"network-check-target-vwpt4\" (UID: \"2494ce49-fb0d-4796-9b59-63dfd2d2c7fd\") " pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:31.726108 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:31.726120 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs\") pod \"network-metrics-daemon-s5vq9\" (UID: \"499e1abf-db52-4ca2-9ea6-208a405209f3\") " pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:31.726349 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:31.726268 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:32:31.726349 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:31.726268 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 20:32:31.726349 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:31.726299 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 20:32:31.726349 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:31.726311 2582 projected.go:196] Error preparing data for projected volume kube-api-access-v5hzv for pod openshift-network-diagnostics/network-check-target-vwpt4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:32:31.726349 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:31.726337 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs podName:499e1abf-db52-4ca2-9ea6-208a405209f3 nodeName:}" failed. No retries permitted until 2026-07-03 20:32:47.726318284 +0000 UTC m=+34.184590715 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs") pod "network-metrics-daemon-s5vq9" (UID: "499e1abf-db52-4ca2-9ea6-208a405209f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:32:31.726581 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:31.726359 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv podName:2494ce49-fb0d-4796-9b59-63dfd2d2c7fd nodeName:}" failed. No retries permitted until 2026-07-03 20:32:47.726344126 +0000 UTC m=+34.184616562 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-v5hzv" (UniqueName: "kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv") pod "network-check-target-vwpt4" (UID: "2494ce49-fb0d-4796-9b59-63dfd2d2c7fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:32:31.980615 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:31.980583 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:31.980817 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:31.980698 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:31.981198 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:31.981080 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:31.981612 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:31.981394 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:32.979942 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:32.979902 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:32.980147 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:32.980054 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:33.981726 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:33.981692 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:33.982168 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:33.981734 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:33.982168 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:33.981833 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:33.982168 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:33.981962 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:34.980683 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:34.980647 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:34.980852 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:34.980767 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:35.980412 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:35.980379 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:35.980978 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:35.980514 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:35.980978 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:35.980558 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:35.980978 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:35.980623 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:36.980233 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:36.979901 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:36.980361 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:36.980335 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:37.119538 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.119511 2582 generic.go:356] "Generic (PLEG): container finished" podID="5609fd01-5426-4ff6-97f5-a3d3367e85f0" containerID="93ebccf2bcc614d6d90a3243ea3be588ea23976c304628e03bd6009ee970f894" exitCode=0 Jul 03 20:32:37.120240 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.119582 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jd92" event={"ID":"5609fd01-5426-4ff6-97f5-a3d3367e85f0","Type":"ContainerDied","Data":"93ebccf2bcc614d6d90a3243ea3be588ea23976c304628e03bd6009ee970f894"} Jul 03 20:32:37.121123 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.121098 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" event={"ID":"cfe616fb-4d9c-4ece-ba6a-c5c58e496738","Type":"ContainerStarted","Data":"48b617064533f479c605612ab4194cf11fe7efcfaae357901587a49a72234ead"} Jul 03 20:32:37.122492 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.122460 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ng2rm" event={"ID":"ee651638-933f-4b70-a0b4-08438004b94d","Type":"ContainerStarted","Data":"60ae5050cc3436ec5e4a64530bf57323d126b7e8a2dcb61e41754cd08a6c6d94"} Jul 03 20:32:37.126328 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.124241 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cztbp" event={"ID":"4483b180-eb1d-488d-a652-4f12c1e41407","Type":"ContainerStarted","Data":"78c41886e07840c723eb27976b7fe00335ea08946b53e61875967a7d05f240fc"} Jul 03 20:32:37.128362 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.128336 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bs4f9" event={"ID":"f6a8fc1e-c013-45e0-a570-97d2e74f7feb","Type":"ContainerStarted","Data":"a571130097d42d7c4771ee8bce38e2a6c032add3e449187dd0c7b2cab9add15f"} Jul 03 20:32:37.129774 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.129723 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4lfd9" event={"ID":"e148c3c4-214b-4ed2-acea-69d3a0fe10b2","Type":"ContainerStarted","Data":"de59b58d65b16fa63f0671471cbb947e587909a8d54cdf8acce4d0299d40005f"} Jul 03 20:32:37.133338 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.133308 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" event={"ID":"f8c1736d-e1e0-4fed-979c-4039a3b4c087","Type":"ContainerStarted","Data":"83d391cf3d161e0a0d59f6f0fcb4c35ae0af740fe9340855e4c4ac67a3a10801"} Jul 03 20:32:37.133432 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.133392 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" event={"ID":"f8c1736d-e1e0-4fed-979c-4039a3b4c087","Type":"ContainerStarted","Data":"cba7c4aed282ec51af93af11955ddaef0d6f1b41951149b66d8c004d5d6769cc"} Jul 03 20:32:37.133493 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.133445 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" event={"ID":"f8c1736d-e1e0-4fed-979c-4039a3b4c087","Type":"ContainerStarted","Data":"dcb50889c62955ae65a81fbb79ca0310966867701c3f419861570a81fed57773"} Jul 03 20:32:37.133552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.133496 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" event={"ID":"f8c1736d-e1e0-4fed-979c-4039a3b4c087","Type":"ContainerStarted","Data":"6623b048740dd21a7c3c7aeddb26462a2ef5493652c1ef73f9f2674e6e5ce1b5"} Jul 03 20:32:37.133552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.133546 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" event={"ID":"f8c1736d-e1e0-4fed-979c-4039a3b4c087","Type":"ContainerStarted","Data":"643f0285f2d6474c00a66cea016d663ae075f6d788a775ecafae8af7fa7ab5f2"} Jul 03 20:32:37.135451 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.135429 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" event={"ID":"e46049a0-7317-495b-8e8d-7663ac8fd178","Type":"ContainerStarted","Data":"3420a27f4cc371601092a8fb60c30fc507b65cff9c9063182e0d35a3a965fc31"} Jul 03 20:32:37.144941 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.144904 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-68.ec2.internal" podStartSLOduration=22.144891915 podStartE2EDuration="22.144891915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:32:19.092202809 +0000 UTC m=+5.550475255" watchObservedRunningTime="2026-07-03 20:32:37.144891915 +0000 UTC m=+23.603164353" Jul 03 20:32:37.158898 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.158848 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ng2rm" podStartSLOduration=11.965514508 podStartE2EDuration="23.158834076s" totalImagesPullingTime="11.193319568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:32:37.158737159 +0000 UTC m=+23.617009597" watchObservedRunningTime="2026-07-03 20:32:37.158834076 +0000 UTC m=+23.617106509" Jul 03 20:32:37.175495 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.175441 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4lfd9" podStartSLOduration=3.791829633 podStartE2EDuration="23.175426094s" totalImagesPullingTime="19.383596461s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:32:37.17524863 +0000 UTC m=+23.633521071" watchObservedRunningTime="2026-07-03 20:32:37.175426094 +0000 UTC m=+23.633698533" Jul 03 20:32:37.191479 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.191423 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8hfsx" podStartSLOduration=3.806311998 podStartE2EDuration="23.191402369s" totalImagesPullingTime="19.385090371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:32:37.19092455 +0000 UTC m=+23.649196991" watchObservedRunningTime="2026-07-03 20:32:37.191402369 +0000 UTC m=+23.649674810" Jul 03 20:32:37.212285 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.212237 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bs4f9" podStartSLOduration=3.811684665 podStartE2EDuration="23.212220838s" totalImagesPullingTime="19.400536173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:32:37.212015929 +0000 UTC m=+23.670288369" watchObservedRunningTime="2026-07-03 20:32:37.212220838 +0000 UTC m=+23.670493335" Jul 03 20:32:37.229449 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.229403 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cztbp" podStartSLOduration=3.774425697 podStartE2EDuration="23.229390094s" totalImagesPullingTime="19.454964397s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:32:37.22884662 +0000 UTC m=+23.687119061" watchObservedRunningTime="2026-07-03 20:32:37.229390094 +0000 UTC m=+23.687662532" Jul 03 20:32:37.723959 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.723925 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 03 20:32:37.950808 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.950572 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-03T20:32:37.723946334Z","UUID":"30660e00-31fe-4877-9275-6a9c0bc09b07","Handler":null,"Name":"","Endpoint":""} Jul 03 20:32:37.952432 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.952406 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 03 20:32:37.952553 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.952454 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 03 20:32:37.979990 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.979952 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:37.980142 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:37.979995 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:37.980142 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:37.980096 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:37.980271 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:37.980241 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:38.139387 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:38.139350 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-th2hs" event={"ID":"3cba7612-a358-44b3-bce9-c3c87215cbcb","Type":"ContainerStarted","Data":"db1f0b392636680f9f05988647a6b20b8ed6650469438c0ebb67edb2afd0f9c3"} Jul 03 20:32:38.142944 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:38.142919 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" event={"ID":"f8c1736d-e1e0-4fed-979c-4039a3b4c087","Type":"ContainerStarted","Data":"06a7e80737f5877e1f46e72880440f9f4443251f1436e9c3bb3a4304583fe25e"} Jul 03 20:32:38.144944 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:38.144900 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" event={"ID":"e46049a0-7317-495b-8e8d-7663ac8fd178","Type":"ContainerStarted","Data":"4d68bc6df7554a555f1ad57425c08d2c79bd9c29008d84ffbf27a7dfa0e9a5b2"} Jul 03 20:32:38.155141 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:38.155085 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-th2hs" podStartSLOduration=4.771049868 podStartE2EDuration="24.155067541s" totalImagesPullingTime="19.384017673s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:32:38.154452508 +0000 UTC m=+24.612724939" watchObservedRunningTime="2026-07-03 20:32:38.155067541 +0000 UTC m=+24.613339980" Jul 03 20:32:38.763019 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:38.762984 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ng2rm" Jul 03 20:32:38.980388 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:38.980360 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:38.980495 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:38.980471 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:39.150271 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:39.150232 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" event={"ID":"e46049a0-7317-495b-8e8d-7663ac8fd178","Type":"ContainerStarted","Data":"b5b848dc2994c42d2f7c4b6b4053f4e78c9b6b10250e2bc8ed40828abaf69c0b"} Jul 03 20:32:39.168966 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:39.168917 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sn9hm" podStartSLOduration=3.393820764 podStartE2EDuration="25.168896598s" totalImagesPullingTime="21.775075834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:32:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:32:39.16794423 +0000 UTC m=+25.626216670" watchObservedRunningTime="2026-07-03 20:32:39.168896598 +0000 UTC m=+25.627169058" Jul 03 20:32:39.980970 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:39.980935 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:39.980970 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:39.980962 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:39.981307 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:39.981076 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:39.981307 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:39.981162 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:40.157137 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:40.157107 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" event={"ID":"f8c1736d-e1e0-4fed-979c-4039a3b4c087","Type":"ContainerStarted","Data":"3938f9a033164098aff99e5746f4fa44770e6d8164788ecacc2a8526cd616bbd"} Jul 03 20:32:40.980654 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:40.980396 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:40.980854 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:40.980770 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:41.962947 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:41.962730 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ng2rm" Jul 03 20:32:41.963665 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:41.963284 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ng2rm" Jul 03 20:32:41.980148 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:41.980119 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:41.980268 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:41.980220 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:41.980319 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:41.980283 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:41.980403 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:41.980384 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:42.164137 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:42.164015 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" event={"ID":"f8c1736d-e1e0-4fed-979c-4039a3b4c087","Type":"ContainerStarted","Data":"3ed93e5203627e907fe4456f259ec21dc99705a8b710dcfed018d42a72545671"} Jul 03 20:32:42.164298 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:42.164256 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:42.165764 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:42.165725 2582 generic.go:356] "Generic (PLEG): container finished" podID="5609fd01-5426-4ff6-97f5-a3d3367e85f0" containerID="8bbb43094c9a544cd5219fa59e117a42f726ccbf57d386c2a1a3064469fe1df8" exitCode=0 Jul 03 20:32:42.165869 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:42.165777 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jd92" event={"ID":"5609fd01-5426-4ff6-97f5-a3d3367e85f0","Type":"ContainerDied","Data":"8bbb43094c9a544cd5219fa59e117a42f726ccbf57d386c2a1a3064469fe1df8"} Jul 03 20:32:42.166542 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:42.166525 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ng2rm" Jul 03 20:32:42.181250 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:42.181232 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:42.189962 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:42.189922 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" podStartSLOduration=8.707152527 podStartE2EDuration="28.189907342s" totalImagesPullingTime="19.482754815s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:32:42.188608559 +0000 UTC m=+28.646880998" watchObservedRunningTime="2026-07-03 20:32:42.189907342 +0000 UTC m=+28.648179781" Jul 03 20:32:42.980726 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:42.980692 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:42.981260 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:42.980854 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:43.170360 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:43.169475 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:43.170360 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:43.169615 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:43.190825 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:43.190675 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:32:43.421084 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:43.421030 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vwpt4"] Jul 03 20:32:43.421457 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:43.421185 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:43.421457 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:43.421297 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:43.421793 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:43.421773 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s5vq9"] Jul 03 20:32:43.421885 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:43.421874 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:43.421976 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:43.421956 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:43.431742 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:43.431705 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-chq7m"] Jul 03 20:32:43.431908 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:43.431842 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:43.431972 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:43.431954 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:44.174248 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:44.174210 2582 generic.go:356] "Generic (PLEG): container finished" podID="5609fd01-5426-4ff6-97f5-a3d3367e85f0" containerID="e260bed142866413f7108840fcbf043024e5b926272616e826171a082b290725" exitCode=0 Jul 03 20:32:44.174617 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:44.174301 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jd92" event={"ID":"5609fd01-5426-4ff6-97f5-a3d3367e85f0","Type":"ContainerDied","Data":"e260bed142866413f7108840fcbf043024e5b926272616e826171a082b290725"} Jul 03 20:32:44.981159 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:44.980966 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:44.981319 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:44.980966 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:44.981319 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:44.981255 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:44.981319 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:44.980966 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:44.981433 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:44.981313 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:44.981433 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:44.981379 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:45.179208 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:45.179162 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jd92" event={"ID":"5609fd01-5426-4ff6-97f5-a3d3367e85f0","Type":"ContainerStarted","Data":"eb98ac7e8b5a2ad71663ac33edfc3726ededf4e9ac602b0f8f3f7d6923b02dc7"} Jul 03 20:32:46.183932 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:46.183903 2582 generic.go:356] "Generic (PLEG): container finished" podID="5609fd01-5426-4ff6-97f5-a3d3367e85f0" containerID="eb98ac7e8b5a2ad71663ac33edfc3726ededf4e9ac602b0f8f3f7d6923b02dc7" exitCode=0 Jul 03 20:32:46.184321 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:46.183947 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jd92" event={"ID":"5609fd01-5426-4ff6-97f5-a3d3367e85f0","Type":"ContainerDied","Data":"eb98ac7e8b5a2ad71663ac33edfc3726ededf4e9ac602b0f8f3f7d6923b02dc7"} Jul 03 20:32:46.980115 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:46.980085 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:46.980307 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:46.980132 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:46.980307 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:46.980208 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:46.980307 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:46.980212 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s5vq9" podUID="499e1abf-db52-4ca2-9ea6-208a405209f3" Jul 03 20:32:46.980447 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:46.980340 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chq7m" podUID="50e502c5-dd61-4c6f-9bcf-6de839bf5fbb" Jul 03 20:32:46.980489 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:46.980469 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwpt4" podUID="2494ce49-fb0d-4796-9b59-63dfd2d2c7fd" Jul 03 20:32:47.646136 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:47.646091 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:47.646633 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:47.646252 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 20:32:47.646633 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:47.646324 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret podName:50e502c5-dd61-4c6f-9bcf-6de839bf5fbb nodeName:}" failed. No retries permitted until 2026-07-03 20:33:19.646307958 +0000 UTC m=+66.104580391 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret") pod "global-pull-secret-syncer-chq7m" (UID: "50e502c5-dd61-4c6f-9bcf-6de839bf5fbb") : object "kube-system"/"original-pull-secret" not registered Jul 03 20:32:47.747311 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:47.747272 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5hzv\" (UniqueName: \"kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv\") pod \"network-check-target-vwpt4\" (UID: \"2494ce49-fb0d-4796-9b59-63dfd2d2c7fd\") " pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:47.747491 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:47.747333 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs\") pod \"network-metrics-daemon-s5vq9\" (UID: \"499e1abf-db52-4ca2-9ea6-208a405209f3\") " pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:47.747491 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:47.747475 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:32:47.747600 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:47.747495 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 20:32:47.747600 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:47.747523 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 20:32:47.747600 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:47.747529 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs podName:499e1abf-db52-4ca2-9ea6-208a405209f3 nodeName:}" failed. No retries permitted until 2026-07-03 20:33:19.747514786 +0000 UTC m=+66.205787203 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs") pod "network-metrics-daemon-s5vq9" (UID: "499e1abf-db52-4ca2-9ea6-208a405209f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:32:47.747600 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:47.747538 2582 projected.go:196] Error preparing data for projected volume kube-api-access-v5hzv for pod openshift-network-diagnostics/network-check-target-vwpt4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:32:47.747600 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:47.747594 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv podName:2494ce49-fb0d-4796-9b59-63dfd2d2c7fd nodeName:}" failed. No retries permitted until 2026-07-03 20:33:19.747576155 +0000 UTC m=+66.205848588 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-v5hzv" (UniqueName: "kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv") pod "network-check-target-vwpt4" (UID: "2494ce49-fb0d-4796-9b59-63dfd2d2c7fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:32:48.902307 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.902276 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-68.ec2.internal" event="NodeReady" Jul 03 20:32:48.902709 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.902447 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 03 20:32:48.939191 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.939157 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5d798d84bc-2plw8"] Jul 03 20:32:48.943798 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.943777 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:48.947519 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.947467 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 03 20:32:48.947635 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.947539 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 03 20:32:48.947635 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.947557 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pk46c\"" Jul 03 20:32:48.947802 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.947686 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 03 20:32:48.953119 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.953095 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 03 20:32:48.954652 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.953793 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rx9p2"] Jul 03 20:32:48.957391 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.957373 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kwwh4"] Jul 03 20:32:48.957530 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.957511 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:48.960076 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.960058 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tfjxf\"" Jul 03 20:32:48.960349 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.960328 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 03 20:32:48.960428 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.960357 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 03 20:32:48.960813 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.960794 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d798d84bc-2plw8"] Jul 03 20:32:48.960892 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.960880 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kwwh4" Jul 03 20:32:48.963087 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.963050 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 03 20:32:48.963319 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.963299 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 03 20:32:48.963420 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.963181 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 03 20:32:48.963420 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.963184 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jqhvm\"" Jul 03 20:32:48.973988 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.973104 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kwwh4"] Jul 03 20:32:48.974107 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.974091 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rx9p2"] Jul 03 20:32:48.980211 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.980189 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:32:48.980320 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.980272 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:32:48.980608 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.980382 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:32:48.982733 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.982713 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 03 20:32:48.983093 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.983076 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9tq75\"" Jul 03 20:32:48.983402 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.983382 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 03 20:32:48.983489 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.983415 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 03 20:32:48.983536 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.983390 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 03 20:32:48.983574 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:48.983382 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k9k4n\"" Jul 03 20:32:49.058721 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.058673 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-bound-sa-token\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.058923 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.058722 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d42b917-3c23-4fa8-8d71-ff9742a87e12-tmp-dir\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:49.058923 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.058831 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-image-registry-private-configuration\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.058923 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.058865 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.058923 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.058890 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8mrk\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-kube-api-access-k8mrk\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.059119 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.058998 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb443c07-c7e6-42e6-9d95-baed044fcdf9-cert\") pod \"ingress-canary-kwwh4\" (UID: \"cb443c07-c7e6-42e6-9d95-baed044fcdf9\") " pod="openshift-ingress-canary/ingress-canary-kwwh4" Jul 03 20:32:49.059119 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.059042 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c9kl\" (UniqueName: \"kubernetes.io/projected/0d42b917-3c23-4fa8-8d71-ff9742a87e12-kube-api-access-8c9kl\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:49.059119 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.059070 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-ca-trust-extracted\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.059119 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.059113 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:49.059269 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.059139 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-certificates\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.059269 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.059197 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fhk5\" (UniqueName: \"kubernetes.io/projected/cb443c07-c7e6-42e6-9d95-baed044fcdf9-kube-api-access-5fhk5\") pod \"ingress-canary-kwwh4\" (UID: \"cb443c07-c7e6-42e6-9d95-baed044fcdf9\") " pod="openshift-ingress-canary/ingress-canary-kwwh4" Jul 03 20:32:49.059269 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.059228 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d42b917-3c23-4fa8-8d71-ff9742a87e12-config-volume\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:49.059269 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.059250 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-installation-pull-secrets\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.059436 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.059270 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-trusted-ca\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.160509 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.160466 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb443c07-c7e6-42e6-9d95-baed044fcdf9-cert\") pod \"ingress-canary-kwwh4\" (UID: \"cb443c07-c7e6-42e6-9d95-baed044fcdf9\") " pod="openshift-ingress-canary/ingress-canary-kwwh4" Jul 03 20:32:49.160509 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.160516 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c9kl\" (UniqueName: \"kubernetes.io/projected/0d42b917-3c23-4fa8-8d71-ff9742a87e12-kube-api-access-8c9kl\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:49.160715 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.160543 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-ca-trust-extracted\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.160715 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.160575 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:49.160715 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.160600 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-certificates\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.160715 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.160634 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fhk5\" (UniqueName: \"kubernetes.io/projected/cb443c07-c7e6-42e6-9d95-baed044fcdf9-kube-api-access-5fhk5\") pod \"ingress-canary-kwwh4\" (UID: \"cb443c07-c7e6-42e6-9d95-baed044fcdf9\") " pod="openshift-ingress-canary/ingress-canary-kwwh4" Jul 03 20:32:49.160715 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.160656 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d42b917-3c23-4fa8-8d71-ff9742a87e12-config-volume\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:49.160999 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:49.160770 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 20:32:49.160999 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:49.160846 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls podName:0d42b917-3c23-4fa8-8d71-ff9742a87e12 nodeName:}" failed. No retries permitted until 2026-07-03 20:32:49.660824248 +0000 UTC m=+36.119096673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls") pod "dns-default-rx9p2" (UID: "0d42b917-3c23-4fa8-8d71-ff9742a87e12") : secret "dns-default-metrics-tls" not found Jul 03 20:32:49.160999 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.160988 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-ca-trust-extracted\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.161147 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.161022 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-installation-pull-secrets\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.161147 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.161060 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-trusted-ca\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.161147 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.161120 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-bound-sa-token\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.161284 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.161151 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d42b917-3c23-4fa8-8d71-ff9742a87e12-tmp-dir\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:49.161284 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.161186 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-image-registry-private-configuration\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.161284 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.161213 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.161284 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.161215 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d42b917-3c23-4fa8-8d71-ff9742a87e12-config-volume\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:49.161284 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.161240 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8mrk\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-kube-api-access-k8mrk\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.161521 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.161458 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-certificates\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.161622 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:49.161588 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 20:32:49.161622 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:49.161615 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d798d84bc-2plw8: secret "image-registry-tls" not found Jul 03 20:32:49.161781 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:49.161696 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls podName:31eaf79b-1cf2-4dcd-8872-d43d5701d9f3 nodeName:}" failed. No retries permitted until 2026-07-03 20:32:49.66167771 +0000 UTC m=+36.119950129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls") pod "image-registry-5d798d84bc-2plw8" (UID: "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3") : secret "image-registry-tls" not found Jul 03 20:32:49.161781 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.161707 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d42b917-3c23-4fa8-8d71-ff9742a87e12-tmp-dir\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:49.162074 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.162050 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-trusted-ca\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.165151 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.165126 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-image-registry-private-configuration\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.165257 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.165200 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-installation-pull-secrets\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.165257 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.165245 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb443c07-c7e6-42e6-9d95-baed044fcdf9-cert\") pod \"ingress-canary-kwwh4\" (UID: \"cb443c07-c7e6-42e6-9d95-baed044fcdf9\") " pod="openshift-ingress-canary/ingress-canary-kwwh4" Jul 03 20:32:49.171015 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.170968 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c9kl\" (UniqueName: \"kubernetes.io/projected/0d42b917-3c23-4fa8-8d71-ff9742a87e12-kube-api-access-8c9kl\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:49.172162 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.172132 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fhk5\" (UniqueName: \"kubernetes.io/projected/cb443c07-c7e6-42e6-9d95-baed044fcdf9-kube-api-access-5fhk5\") pod \"ingress-canary-kwwh4\" (UID: \"cb443c07-c7e6-42e6-9d95-baed044fcdf9\") " pod="openshift-ingress-canary/ingress-canary-kwwh4" Jul 03 20:32:49.172268 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.172163 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8mrk\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-kube-api-access-k8mrk\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.172982 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.172960 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-bound-sa-token\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.276955 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.276863 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kwwh4" Jul 03 20:32:49.666040 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.666003 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:49.666224 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:49.666097 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:49.666224 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:49.666177 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 20:32:49.666224 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:49.666201 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d798d84bc-2plw8: secret "image-registry-tls" not found Jul 03 20:32:49.666224 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:49.666224 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 20:32:49.666401 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:49.666261 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls podName:31eaf79b-1cf2-4dcd-8872-d43d5701d9f3 nodeName:}" failed. No retries permitted until 2026-07-03 20:32:50.666242975 +0000 UTC m=+37.124515393 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls") pod "image-registry-5d798d84bc-2plw8" (UID: "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3") : secret "image-registry-tls" not found Jul 03 20:32:49.666401 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:49.666279 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls podName:0d42b917-3c23-4fa8-8d71-ff9742a87e12 nodeName:}" failed. No retries permitted until 2026-07-03 20:32:50.666270858 +0000 UTC m=+37.124543278 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls") pod "dns-default-rx9p2" (UID: "0d42b917-3c23-4fa8-8d71-ff9742a87e12") : secret "dns-default-metrics-tls" not found Jul 03 20:32:50.221281 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:50.221091 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kwwh4"] Jul 03 20:32:50.226397 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:50.226368 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb443c07_c7e6_42e6_9d95_baed044fcdf9.slice/crio-dd70ff78af6f4f89ecf468dfbff91c149e6abc1e0159926841880f68d438af19 WatchSource:0}: Error finding container dd70ff78af6f4f89ecf468dfbff91c149e6abc1e0159926841880f68d438af19: Status 404 returned error can't find the container with id dd70ff78af6f4f89ecf468dfbff91c149e6abc1e0159926841880f68d438af19 Jul 03 20:32:50.673704 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:50.673666 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:50.673943 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:50.673733 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:50.673943 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:50.673857 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 20:32:50.673943 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:50.673862 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 20:32:50.673943 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:50.673888 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d798d84bc-2plw8: secret "image-registry-tls" not found Jul 03 20:32:50.673943 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:50.673937 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls podName:0d42b917-3c23-4fa8-8d71-ff9742a87e12 nodeName:}" failed. No retries permitted until 2026-07-03 20:32:52.673916984 +0000 UTC m=+39.132189403 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls") pod "dns-default-rx9p2" (UID: "0d42b917-3c23-4fa8-8d71-ff9742a87e12") : secret "dns-default-metrics-tls" not found Jul 03 20:32:50.674131 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:50.673958 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls podName:31eaf79b-1cf2-4dcd-8872-d43d5701d9f3 nodeName:}" failed. No retries permitted until 2026-07-03 20:32:52.673948577 +0000 UTC m=+39.132221001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls") pod "image-registry-5d798d84bc-2plw8" (UID: "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3") : secret "image-registry-tls" not found Jul 03 20:32:51.199576 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:51.199539 2582 generic.go:356] "Generic (PLEG): container finished" podID="5609fd01-5426-4ff6-97f5-a3d3367e85f0" containerID="2fb9a67c563f5928830f337a48d715d0c6fe0ba3c1be73b867a3221f37dae594" exitCode=0 Jul 03 20:32:51.199769 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:51.199621 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jd92" event={"ID":"5609fd01-5426-4ff6-97f5-a3d3367e85f0","Type":"ContainerDied","Data":"2fb9a67c563f5928830f337a48d715d0c6fe0ba3c1be73b867a3221f37dae594"} Jul 03 20:32:51.200901 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:51.200873 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kwwh4" event={"ID":"cb443c07-c7e6-42e6-9d95-baed044fcdf9","Type":"ContainerStarted","Data":"dd70ff78af6f4f89ecf468dfbff91c149e6abc1e0159926841880f68d438af19"} Jul 03 20:32:52.207439 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:52.207391 2582 generic.go:356] "Generic (PLEG): container finished" podID="5609fd01-5426-4ff6-97f5-a3d3367e85f0" containerID="06609313f83b0b8623bd17ac23ab8ba8fa18e301a8b44451a6e330394e85b600" exitCode=0 Jul 03 20:32:52.207858 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:52.207452 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jd92" event={"ID":"5609fd01-5426-4ff6-97f5-a3d3367e85f0","Type":"ContainerDied","Data":"06609313f83b0b8623bd17ac23ab8ba8fa18e301a8b44451a6e330394e85b600"} Jul 03 20:32:52.690085 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:52.689903 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:52.690085 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:52.690068 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 20:32:52.690085 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:52.690087 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d798d84bc-2plw8: secret "image-registry-tls" not found Jul 03 20:32:52.690392 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:52.690123 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:52.690392 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:52.690140 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls podName:31eaf79b-1cf2-4dcd-8872-d43d5701d9f3 nodeName:}" failed. No retries permitted until 2026-07-03 20:32:56.690124878 +0000 UTC m=+43.148397296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls") pod "image-registry-5d798d84bc-2plw8" (UID: "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3") : secret "image-registry-tls" not found Jul 03 20:32:52.690392 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:52.690192 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 20:32:52.690392 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:52.690245 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls podName:0d42b917-3c23-4fa8-8d71-ff9742a87e12 nodeName:}" failed. No retries permitted until 2026-07-03 20:32:56.690231689 +0000 UTC m=+43.148504123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls") pod "dns-default-rx9p2" (UID: "0d42b917-3c23-4fa8-8d71-ff9742a87e12") : secret "dns-default-metrics-tls" not found Jul 03 20:32:53.213305 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:53.213272 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7jd92" event={"ID":"5609fd01-5426-4ff6-97f5-a3d3367e85f0","Type":"ContainerStarted","Data":"0d908c00438a0c4f03b8df6cce66922c12c17550cacf4ed343099685f51c4c21"} Jul 03 20:32:53.214767 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:53.214728 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kwwh4" event={"ID":"cb443c07-c7e6-42e6-9d95-baed044fcdf9","Type":"ContainerStarted","Data":"409253f7dadf5f80bbf0feb199e3b7c62cc7908f885e519c61e387171c784d56"} Jul 03 20:32:53.258812 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:53.258697 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7jd92" podStartSLOduration=9.881741222 podStartE2EDuration="39.258683059s" totalImagesPullingTime="29.376941837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:32:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:32:53.258372245 +0000 UTC m=+39.716644696" watchObservedRunningTime="2026-07-03 20:32:53.258683059 +0000 UTC m=+39.716955497" Jul 03 20:32:53.285026 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:53.284984 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kwwh4" podStartSLOduration=3.054393675 podStartE2EDuration="5.284970797s" totalImagesPullingTime="2.230577122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:32:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:32:53.284643723 +0000 UTC m=+39.742916178" watchObservedRunningTime="2026-07-03 20:32:53.284970797 +0000 UTC m=+39.743243263" Jul 03 20:32:54.518535 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.518502 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v7j67"] Jul 03 20:32:54.542544 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.542518 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v7j67"] Jul 03 20:32:54.542685 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.542627 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:54.545292 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.545270 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 03 20:32:54.545292 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.545280 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-498s6\"" Jul 03 20:32:54.546173 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.546156 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 03 20:32:54.546173 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.546171 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 03 20:32:54.546367 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.546160 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 03 20:32:54.706567 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.706528 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjb6t\" (UniqueName: \"kubernetes.io/projected/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-api-access-xjb6t\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:54.706707 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.706607 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:54.706707 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.706627 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:54.706707 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.706652 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2004c927-d12e-4dc3-9c28-01d6e488fc0e-data-volume\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:54.706707 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.706668 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2004c927-d12e-4dc3-9c28-01d6e488fc0e-crio-socket\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:54.807796 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.807701 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2004c927-d12e-4dc3-9c28-01d6e488fc0e-crio-socket\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:54.807796 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.807789 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjb6t\" (UniqueName: \"kubernetes.io/projected/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-api-access-xjb6t\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:54.807951 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.807890 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:54.807951 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.807913 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:54.807951 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.807947 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2004c927-d12e-4dc3-9c28-01d6e488fc0e-data-volume\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:54.808098 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.807958 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2004c927-d12e-4dc3-9c28-01d6e488fc0e-crio-socket\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:54.808098 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:54.808037 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 20:32:54.808098 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:54.808075 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 20:32:54.808220 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:54.808105 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls podName:2004c927-d12e-4dc3-9c28-01d6e488fc0e nodeName:}" failed. No retries permitted until 2026-07-03 20:32:55.308089168 +0000 UTC m=+41.766361588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v7j67" (UID: "2004c927-d12e-4dc3-9c28-01d6e488fc0e") : secret "insights-runtime-extractor-tls" not found Jul 03 20:32:54.808220 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:54.808149 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm podName:2004c927-d12e-4dc3-9c28-01d6e488fc0e nodeName:}" failed. No retries permitted until 2026-07-03 20:32:55.308130106 +0000 UTC m=+41.766402544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v7j67" (UID: "2004c927-d12e-4dc3-9c28-01d6e488fc0e") : configmap "kube-rbac-proxy" not found Jul 03 20:32:54.808312 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.808245 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2004c927-d12e-4dc3-9c28-01d6e488fc0e-data-volume\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:54.820142 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:54.820123 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjb6t\" (UniqueName: \"kubernetes.io/projected/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-api-access-xjb6t\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:55.310934 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:55.310905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:55.310934 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:55.310938 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:55.311142 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:55.311002 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 20:32:55.311142 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:55.311050 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm podName:2004c927-d12e-4dc3-9c28-01d6e488fc0e nodeName:}" failed. No retries permitted until 2026-07-03 20:32:56.311036994 +0000 UTC m=+42.769309411 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v7j67" (UID: "2004c927-d12e-4dc3-9c28-01d6e488fc0e") : configmap "kube-rbac-proxy" not found Jul 03 20:32:55.311142 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:55.311054 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 20:32:55.311142 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:55.311103 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls podName:2004c927-d12e-4dc3-9c28-01d6e488fc0e nodeName:}" failed. No retries permitted until 2026-07-03 20:32:56.311090805 +0000 UTC m=+42.769363266 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v7j67" (UID: "2004c927-d12e-4dc3-9c28-01d6e488fc0e") : secret "insights-runtime-extractor-tls" not found Jul 03 20:32:55.387310 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:55.387234 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-xfk8t"] Jul 03 20:32:55.418281 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:55.418256 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-xfk8t"] Jul 03 20:32:55.418414 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:55.418350 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xfk8t" Jul 03 20:32:55.420923 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:55.420903 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-dd26w\"" Jul 03 20:32:55.421014 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:55.420993 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 03 20:32:55.421350 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:55.421335 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 03 20:32:55.512217 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:55.512189 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlv5b\" (UniqueName: \"kubernetes.io/projected/a7456447-e65b-4775-bec1-822fb72392c8-kube-api-access-rlv5b\") pod \"migrator-d57d58cc9-xfk8t\" (UID: \"a7456447-e65b-4775-bec1-822fb72392c8\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xfk8t" Jul 03 20:32:55.613591 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:55.613553 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlv5b\" (UniqueName: \"kubernetes.io/projected/a7456447-e65b-4775-bec1-822fb72392c8-kube-api-access-rlv5b\") pod \"migrator-d57d58cc9-xfk8t\" (UID: \"a7456447-e65b-4775-bec1-822fb72392c8\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xfk8t" Jul 03 20:32:55.624213 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:55.624188 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlv5b\" (UniqueName: \"kubernetes.io/projected/a7456447-e65b-4775-bec1-822fb72392c8-kube-api-access-rlv5b\") pod \"migrator-d57d58cc9-xfk8t\" (UID: \"a7456447-e65b-4775-bec1-822fb72392c8\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xfk8t" Jul 03 20:32:55.726339 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:55.726312 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xfk8t" Jul 03 20:32:55.866614 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:55.866585 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-xfk8t"] Jul 03 20:32:55.869620 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:55.869589 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7456447_e65b_4775_bec1_822fb72392c8.slice/crio-8615dab76ed7b3acf6e6b932f847d7a9c46b06c23bdac3b0e7dcf91c6b387902 WatchSource:0}: Error finding container 8615dab76ed7b3acf6e6b932f847d7a9c46b06c23bdac3b0e7dcf91c6b387902: Status 404 returned error can't find the container with id 8615dab76ed7b3acf6e6b932f847d7a9c46b06c23bdac3b0e7dcf91c6b387902 Jul 03 20:32:55.901629 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:55.901607 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4lfd9_e148c3c4-214b-4ed2-acea-69d3a0fe10b2/dns-node-resolver/0.log" Jul 03 20:32:56.223598 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:56.223560 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xfk8t" event={"ID":"a7456447-e65b-4775-bec1-822fb72392c8","Type":"ContainerStarted","Data":"8615dab76ed7b3acf6e6b932f847d7a9c46b06c23bdac3b0e7dcf91c6b387902"} Jul 03 20:32:56.319841 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:56.319805 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:56.319841 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:56.319843 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:56.320051 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:56.319956 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 20:32:56.320051 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:56.319966 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 20:32:56.320051 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:56.320013 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm podName:2004c927-d12e-4dc3-9c28-01d6e488fc0e nodeName:}" failed. No retries permitted until 2026-07-03 20:32:58.319999683 +0000 UTC m=+44.778272099 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v7j67" (UID: "2004c927-d12e-4dc3-9c28-01d6e488fc0e") : configmap "kube-rbac-proxy" not found Jul 03 20:32:56.320051 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:56.320028 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls podName:2004c927-d12e-4dc3-9c28-01d6e488fc0e nodeName:}" failed. No retries permitted until 2026-07-03 20:32:58.320021623 +0000 UTC m=+44.778294041 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v7j67" (UID: "2004c927-d12e-4dc3-9c28-01d6e488fc0e") : secret "insights-runtime-extractor-tls" not found Jul 03 20:32:56.723455 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:56.723421 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:32:56.723890 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:56.723485 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:32:56.723890 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:56.723594 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 20:32:56.723890 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:56.723608 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 20:32:56.723890 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:56.723631 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d798d84bc-2plw8: secret "image-registry-tls" not found Jul 03 20:32:56.723890 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:56.723650 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls podName:0d42b917-3c23-4fa8-8d71-ff9742a87e12 nodeName:}" failed. No retries permitted until 2026-07-03 20:33:04.723634088 +0000 UTC m=+51.181906529 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls") pod "dns-default-rx9p2" (UID: "0d42b917-3c23-4fa8-8d71-ff9742a87e12") : secret "dns-default-metrics-tls" not found Jul 03 20:32:56.723890 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:56.723710 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls podName:31eaf79b-1cf2-4dcd-8872-d43d5701d9f3 nodeName:}" failed. No retries permitted until 2026-07-03 20:33:04.723692205 +0000 UTC m=+51.181964623 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls") pod "image-registry-5d798d84bc-2plw8" (UID: "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3") : secret "image-registry-tls" not found Jul 03 20:32:56.901224 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:56.901198 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bs4f9_f6a8fc1e-c013-45e0-a570-97d2e74f7feb/node-ca/0.log" Jul 03 20:32:57.052008 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.051936 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-4mfgs"] Jul 03 20:32:57.064615 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.064582 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-4mfgs" Jul 03 20:32:57.065718 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.065691 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-4mfgs"] Jul 03 20:32:57.067004 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.066981 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 03 20:32:57.067123 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.067082 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 03 20:32:57.068013 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.067996 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-mfx5m\"" Jul 03 20:32:57.068115 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.068042 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 03 20:32:57.068115 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.068108 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 03 20:32:57.228597 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.228562 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74j54\" (UniqueName: \"kubernetes.io/projected/b50c03f5-7e5f-4df6-8c96-041fe4ad345e-kube-api-access-74j54\") pod \"service-ca-758f599c5-4mfgs\" (UID: \"b50c03f5-7e5f-4df6-8c96-041fe4ad345e\") " pod="openshift-service-ca/service-ca-758f599c5-4mfgs" Jul 03 20:32:57.228768 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.228610 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b50c03f5-7e5f-4df6-8c96-041fe4ad345e-signing-key\") pod \"service-ca-758f599c5-4mfgs\" (UID: \"b50c03f5-7e5f-4df6-8c96-041fe4ad345e\") " pod="openshift-service-ca/service-ca-758f599c5-4mfgs" Jul 03 20:32:57.228768 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.228695 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b50c03f5-7e5f-4df6-8c96-041fe4ad345e-signing-cabundle\") pod \"service-ca-758f599c5-4mfgs\" (UID: \"b50c03f5-7e5f-4df6-8c96-041fe4ad345e\") " pod="openshift-service-ca/service-ca-758f599c5-4mfgs" Jul 03 20:32:57.330085 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.330054 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74j54\" (UniqueName: \"kubernetes.io/projected/b50c03f5-7e5f-4df6-8c96-041fe4ad345e-kube-api-access-74j54\") pod \"service-ca-758f599c5-4mfgs\" (UID: \"b50c03f5-7e5f-4df6-8c96-041fe4ad345e\") " pod="openshift-service-ca/service-ca-758f599c5-4mfgs" Jul 03 20:32:57.330191 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.330120 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b50c03f5-7e5f-4df6-8c96-041fe4ad345e-signing-key\") pod \"service-ca-758f599c5-4mfgs\" (UID: \"b50c03f5-7e5f-4df6-8c96-041fe4ad345e\") " pod="openshift-service-ca/service-ca-758f599c5-4mfgs" Jul 03 20:32:57.330191 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.330175 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b50c03f5-7e5f-4df6-8c96-041fe4ad345e-signing-cabundle\") pod \"service-ca-758f599c5-4mfgs\" (UID: \"b50c03f5-7e5f-4df6-8c96-041fe4ad345e\") " pod="openshift-service-ca/service-ca-758f599c5-4mfgs" Jul 03 20:32:57.330842 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.330822 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b50c03f5-7e5f-4df6-8c96-041fe4ad345e-signing-cabundle\") pod \"service-ca-758f599c5-4mfgs\" (UID: \"b50c03f5-7e5f-4df6-8c96-041fe4ad345e\") " pod="openshift-service-ca/service-ca-758f599c5-4mfgs" Jul 03 20:32:57.332652 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.332635 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b50c03f5-7e5f-4df6-8c96-041fe4ad345e-signing-key\") pod \"service-ca-758f599c5-4mfgs\" (UID: \"b50c03f5-7e5f-4df6-8c96-041fe4ad345e\") " pod="openshift-service-ca/service-ca-758f599c5-4mfgs" Jul 03 20:32:57.338471 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.338454 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74j54\" (UniqueName: \"kubernetes.io/projected/b50c03f5-7e5f-4df6-8c96-041fe4ad345e-kube-api-access-74j54\") pod \"service-ca-758f599c5-4mfgs\" (UID: \"b50c03f5-7e5f-4df6-8c96-041fe4ad345e\") " pod="openshift-service-ca/service-ca-758f599c5-4mfgs" Jul 03 20:32:57.374276 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.374240 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-4mfgs" Jul 03 20:32:57.526979 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.526950 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-4mfgs"] Jul 03 20:32:57.530015 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:32:57.529993 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb50c03f5_7e5f_4df6_8c96_041fe4ad345e.slice/crio-68ef6d0522a1f6e91daf3d8ac7fe4e3f48fd2a5b0844b480f42b43b477f475f6 WatchSource:0}: Error finding container 68ef6d0522a1f6e91daf3d8ac7fe4e3f48fd2a5b0844b480f42b43b477f475f6: Status 404 returned error can't find the container with id 68ef6d0522a1f6e91daf3d8ac7fe4e3f48fd2a5b0844b480f42b43b477f475f6 Jul 03 20:32:57.701098 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:57.701071 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kwwh4_cb443c07-c7e6-42e6-9d95-baed044fcdf9/serve-healthcheck-canary/0.log" Jul 03 20:32:58.229740 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:58.229672 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-4mfgs" event={"ID":"b50c03f5-7e5f-4df6-8c96-041fe4ad345e","Type":"ContainerStarted","Data":"68ef6d0522a1f6e91daf3d8ac7fe4e3f48fd2a5b0844b480f42b43b477f475f6"} Jul 03 20:32:58.231940 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:58.231902 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xfk8t" event={"ID":"a7456447-e65b-4775-bec1-822fb72392c8","Type":"ContainerStarted","Data":"933d3e18afcb8f159f82647285f87978c7f98653bd2cf85d7898dd35969275d8"} Jul 03 20:32:58.232040 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:58.231954 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xfk8t" event={"ID":"a7456447-e65b-4775-bec1-822fb72392c8","Type":"ContainerStarted","Data":"a01cef0c7bde2739cb879be573d92eaa4bd149822a492539c707081521e009b4"} Jul 03 20:32:58.254638 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:58.254582 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xfk8t" podStartSLOduration=1.834720086 podStartE2EDuration="3.254565843s" totalImagesPullingTime="1.419845757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:32:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:32:58.253911539 +0000 UTC m=+44.712183978" watchObservedRunningTime="2026-07-03 20:32:58.254565843 +0000 UTC m=+44.712838278" Jul 03 20:32:58.339041 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:58.339007 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:58.339041 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:32:58.339045 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:32:58.339293 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:58.339170 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 20:32:58.339293 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:58.339239 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls podName:2004c927-d12e-4dc3-9c28-01d6e488fc0e nodeName:}" failed. No retries permitted until 2026-07-03 20:33:02.339220974 +0000 UTC m=+48.797493392 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v7j67" (UID: "2004c927-d12e-4dc3-9c28-01d6e488fc0e") : secret "insights-runtime-extractor-tls" not found Jul 03 20:32:58.339504 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:58.339478 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 20:32:58.339592 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:32:58.339557 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm podName:2004c927-d12e-4dc3-9c28-01d6e488fc0e nodeName:}" failed. No retries permitted until 2026-07-03 20:33:02.33953746 +0000 UTC m=+48.797809882 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v7j67" (UID: "2004c927-d12e-4dc3-9c28-01d6e488fc0e") : configmap "kube-rbac-proxy" not found Jul 03 20:33:00.237550 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:00.237511 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-4mfgs" event={"ID":"b50c03f5-7e5f-4df6-8c96-041fe4ad345e","Type":"ContainerStarted","Data":"5bb0eb05a276ada0e67593e7c37cec854b6f90028ea02894334a1f4bf4890b2b"} Jul 03 20:33:00.257838 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:00.257787 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-4mfgs" podStartSLOduration=1.157404383 podStartE2EDuration="3.257771223s" totalImagesPullingTime="2.10036684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:32:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:33:00.256572188 +0000 UTC m=+46.714844628" watchObservedRunningTime="2026-07-03 20:33:00.257771223 +0000 UTC m=+46.716043662" Jul 03 20:33:02.372783 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:02.372725 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:33:02.372783 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:02.372788 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:33:02.373238 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:33:02.372908 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 20:33:02.373238 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:33:02.372992 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls podName:2004c927-d12e-4dc3-9c28-01d6e488fc0e nodeName:}" failed. No retries permitted until 2026-07-03 20:33:10.372971814 +0000 UTC m=+56.831244234 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v7j67" (UID: "2004c927-d12e-4dc3-9c28-01d6e488fc0e") : secret "insights-runtime-extractor-tls" not found Jul 03 20:33:02.373326 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:02.373259 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2004c927-d12e-4dc3-9c28-01d6e488fc0e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:33:04.794170 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:04.794134 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:33:04.794552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:04.794207 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:33:04.796695 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:04.796668 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d42b917-3c23-4fa8-8d71-ff9742a87e12-metrics-tls\") pod \"dns-default-rx9p2\" (UID: \"0d42b917-3c23-4fa8-8d71-ff9742a87e12\") " pod="openshift-dns/dns-default-rx9p2" Jul 03 20:33:04.796874 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:04.796852 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls\") pod \"image-registry-5d798d84bc-2plw8\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:33:04.855812 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:04.855788 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:33:04.869925 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:04.869902 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rx9p2" Jul 03 20:33:05.029389 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:05.029362 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d798d84bc-2plw8"] Jul 03 20:33:05.032282 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:33:05.032250 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31eaf79b_1cf2_4dcd_8872_d43d5701d9f3.slice/crio-bbd173253d587c426dc346fb24c56a6cf3a9f3afc779e3af9111260614075cbd WatchSource:0}: Error finding container bbd173253d587c426dc346fb24c56a6cf3a9f3afc779e3af9111260614075cbd: Status 404 returned error can't find the container with id bbd173253d587c426dc346fb24c56a6cf3a9f3afc779e3af9111260614075cbd Jul 03 20:33:05.035802 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:05.035783 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rx9p2"] Jul 03 20:33:05.038326 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:33:05.038303 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d42b917_3c23_4fa8_8d71_ff9742a87e12.slice/crio-0e7fc3bd3c7b920ab10e892cd9dba16a0510ce37239153f730f6ade90308201f WatchSource:0}: Error finding container 0e7fc3bd3c7b920ab10e892cd9dba16a0510ce37239153f730f6ade90308201f: Status 404 returned error can't find the container with id 0e7fc3bd3c7b920ab10e892cd9dba16a0510ce37239153f730f6ade90308201f Jul 03 20:33:05.251776 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:05.251730 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rx9p2" event={"ID":"0d42b917-3c23-4fa8-8d71-ff9742a87e12","Type":"ContainerStarted","Data":"0e7fc3bd3c7b920ab10e892cd9dba16a0510ce37239153f730f6ade90308201f"} Jul 03 20:33:05.253182 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:05.253162 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" event={"ID":"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3","Type":"ContainerStarted","Data":"632d96d11139e6daa6d9994b853a9877c7939e2670341d2ec47b060116795f2d"} Jul 03 20:33:05.253256 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:05.253199 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" event={"ID":"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3","Type":"ContainerStarted","Data":"bbd173253d587c426dc346fb24c56a6cf3a9f3afc779e3af9111260614075cbd"} Jul 03 20:33:05.253307 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:05.253293 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:33:05.281856 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:05.281811 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" podStartSLOduration=39.281796671 podStartE2EDuration="39.281796671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:32:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:33:05.280673744 +0000 UTC m=+51.738946183" watchObservedRunningTime="2026-07-03 20:33:05.281796671 +0000 UTC m=+51.740069119" Jul 03 20:33:07.261324 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:07.261293 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rx9p2" event={"ID":"0d42b917-3c23-4fa8-8d71-ff9742a87e12","Type":"ContainerStarted","Data":"9a06af41ac46cd790187d8c0448abd5f910b5f470f20968f18a248692a861a28"} Jul 03 20:33:08.266088 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:08.266049 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rx9p2" event={"ID":"0d42b917-3c23-4fa8-8d71-ff9742a87e12","Type":"ContainerStarted","Data":"25a3e10ac20374f4afe2927a7fb303222429eac247bd6f36f162c5c6aa04f1af"} Jul 03 20:33:08.266463 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:08.266176 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rx9p2" Jul 03 20:33:08.290366 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:08.290322 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rx9p2" podStartSLOduration=18.312342285 podStartE2EDuration="20.290308706s" totalImagesPullingTime="1.977966421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:32:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:33:08.289354356 +0000 UTC m=+54.747626796" watchObservedRunningTime="2026-07-03 20:33:08.290308706 +0000 UTC m=+54.748581144" Jul 03 20:33:10.441508 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:10.441470 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:33:10.441902 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:33:10.441585 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 20:33:10.441902 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:33:10.441648 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls podName:2004c927-d12e-4dc3-9c28-01d6e488fc0e nodeName:}" failed. No retries permitted until 2026-07-03 20:33:26.441633905 +0000 UTC m=+72.899906322 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v7j67" (UID: "2004c927-d12e-4dc3-9c28-01d6e488fc0e") : secret "insights-runtime-extractor-tls" not found Jul 03 20:33:15.193820 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:15.193790 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j8gc9" Jul 03 20:33:18.139944 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.139911 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d798d84bc-2plw8"] Jul 03 20:33:18.147450 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.147420 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf"] Jul 03 20:33:18.150573 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.150546 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" Jul 03 20:33:18.152044 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.152023 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d"] Jul 03 20:33:18.154050 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.154036 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz"] Jul 03 20:33:18.154271 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.154230 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.154380 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.154346 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 03 20:33:18.154445 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.154402 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 03 20:33:18.154445 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.154431 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 03 20:33:18.154662 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.154645 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 03 20:33:18.156024 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.156011 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz" Jul 03 20:33:18.156515 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.156494 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 03 20:33:18.157217 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.157200 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 03 20:33:18.157294 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.157202 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 03 20:33:18.158964 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.158932 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 03 20:33:18.159159 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.159144 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 03 20:33:18.159245 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.159231 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-l8269\"" Jul 03 20:33:18.163963 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.163943 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf"] Jul 03 20:33:18.168019 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.168000 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz"] Jul 03 20:33:18.169028 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.169009 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d"] Jul 03 20:33:18.200523 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.200498 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/46f0ed82-54bb-4065-832f-07146828a892-hub\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.200694 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.200536 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tsrq\" (UniqueName: \"kubernetes.io/projected/db5957a1-4bb1-4f2e-8199-c9fecb20a624-kube-api-access-5tsrq\") pod \"klusterlet-addon-workmgr-77b6ffb954-8gkvf\" (UID: \"db5957a1-4bb1-4f2e-8199-c9fecb20a624\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" Jul 03 20:33:18.200694 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.200558 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/46f0ed82-54bb-4065-832f-07146828a892-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.200694 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.200604 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db5957a1-4bb1-4f2e-8199-c9fecb20a624-tmp\") pod \"klusterlet-addon-workmgr-77b6ffb954-8gkvf\" (UID: \"db5957a1-4bb1-4f2e-8199-c9fecb20a624\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" Jul 03 20:33:18.200694 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.200678 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/46f0ed82-54bb-4065-832f-07146828a892-ca\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.200917 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.200714 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxx5l\" (UniqueName: \"kubernetes.io/projected/46f0ed82-54bb-4065-832f-07146828a892-kube-api-access-wxx5l\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.200917 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.200769 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv2r2\" (UniqueName: \"kubernetes.io/projected/6b72694b-08b9-445f-8129-60b0de44040a-kube-api-access-gv2r2\") pod \"managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz\" (UID: \"6b72694b-08b9-445f-8129-60b0de44040a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz" Jul 03 20:33:18.200917 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.200796 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/46f0ed82-54bb-4065-832f-07146828a892-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.200917 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.200861 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b72694b-08b9-445f-8129-60b0de44040a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz\" (UID: \"6b72694b-08b9-445f-8129-60b0de44040a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz" Jul 03 20:33:18.201045 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.200918 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/46f0ed82-54bb-4065-832f-07146828a892-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.201045 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.200971 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/db5957a1-4bb1-4f2e-8199-c9fecb20a624-klusterlet-config\") pod \"klusterlet-addon-workmgr-77b6ffb954-8gkvf\" (UID: \"db5957a1-4bb1-4f2e-8199-c9fecb20a624\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" Jul 03 20:33:18.267666 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.267632 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-684848dfc7-kqrbk"] Jul 03 20:33:18.269929 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.269898 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.271677 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.271657 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rx9p2" Jul 03 20:33:18.300711 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.300686 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-684848dfc7-kqrbk"] Jul 03 20:33:18.301452 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.301435 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/db5957a1-4bb1-4f2e-8199-c9fecb20a624-klusterlet-config\") pod \"klusterlet-addon-workmgr-77b6ffb954-8gkvf\" (UID: \"db5957a1-4bb1-4f2e-8199-c9fecb20a624\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" Jul 03 20:33:18.301522 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.301470 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/46f0ed82-54bb-4065-832f-07146828a892-hub\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.301522 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.301501 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tsrq\" (UniqueName: \"kubernetes.io/projected/db5957a1-4bb1-4f2e-8199-c9fecb20a624-kube-api-access-5tsrq\") pod \"klusterlet-addon-workmgr-77b6ffb954-8gkvf\" (UID: \"db5957a1-4bb1-4f2e-8199-c9fecb20a624\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" Jul 03 20:33:18.301585 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.301529 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/46f0ed82-54bb-4065-832f-07146828a892-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.301585 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.301557 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db5957a1-4bb1-4f2e-8199-c9fecb20a624-tmp\") pod \"klusterlet-addon-workmgr-77b6ffb954-8gkvf\" (UID: \"db5957a1-4bb1-4f2e-8199-c9fecb20a624\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" Jul 03 20:33:18.301678 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.301585 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/46f0ed82-54bb-4065-832f-07146828a892-ca\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.301678 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.301663 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxx5l\" (UniqueName: \"kubernetes.io/projected/46f0ed82-54bb-4065-832f-07146828a892-kube-api-access-wxx5l\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.301787 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.301710 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv2r2\" (UniqueName: \"kubernetes.io/projected/6b72694b-08b9-445f-8129-60b0de44040a-kube-api-access-gv2r2\") pod \"managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz\" (UID: \"6b72694b-08b9-445f-8129-60b0de44040a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz" Jul 03 20:33:18.301787 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.301736 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/46f0ed82-54bb-4065-832f-07146828a892-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.301889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.301803 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b72694b-08b9-445f-8129-60b0de44040a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz\" (UID: \"6b72694b-08b9-445f-8129-60b0de44040a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz" Jul 03 20:33:18.301957 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.301904 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/46f0ed82-54bb-4065-832f-07146828a892-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.302425 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.302395 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db5957a1-4bb1-4f2e-8199-c9fecb20a624-tmp\") pod \"klusterlet-addon-workmgr-77b6ffb954-8gkvf\" (UID: \"db5957a1-4bb1-4f2e-8199-c9fecb20a624\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" Jul 03 20:33:18.303038 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.303011 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/46f0ed82-54bb-4065-832f-07146828a892-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.305347 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.305287 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/db5957a1-4bb1-4f2e-8199-c9fecb20a624-klusterlet-config\") pod \"klusterlet-addon-workmgr-77b6ffb954-8gkvf\" (UID: \"db5957a1-4bb1-4f2e-8199-c9fecb20a624\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" Jul 03 20:33:18.305698 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.305652 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/46f0ed82-54bb-4065-832f-07146828a892-ca\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.305806 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.305726 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/46f0ed82-54bb-4065-832f-07146828a892-hub\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.305806 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.305768 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/46f0ed82-54bb-4065-832f-07146828a892-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.305907 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.305825 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6b72694b-08b9-445f-8129-60b0de44040a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz\" (UID: \"6b72694b-08b9-445f-8129-60b0de44040a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz" Jul 03 20:33:18.306067 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.306043 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/46f0ed82-54bb-4065-832f-07146828a892-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.324848 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.324811 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxx5l\" (UniqueName: \"kubernetes.io/projected/46f0ed82-54bb-4065-832f-07146828a892-kube-api-access-wxx5l\") pod \"cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d\" (UID: \"46f0ed82-54bb-4065-832f-07146828a892\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.324981 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.324855 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv2r2\" (UniqueName: \"kubernetes.io/projected/6b72694b-08b9-445f-8129-60b0de44040a-kube-api-access-gv2r2\") pod \"managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz\" (UID: \"6b72694b-08b9-445f-8129-60b0de44040a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz" Jul 03 20:33:18.325074 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.325057 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tsrq\" (UniqueName: \"kubernetes.io/projected/db5957a1-4bb1-4f2e-8199-c9fecb20a624-kube-api-access-5tsrq\") pod \"klusterlet-addon-workmgr-77b6ffb954-8gkvf\" (UID: \"db5957a1-4bb1-4f2e-8199-c9fecb20a624\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" Jul 03 20:33:18.402811 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.402778 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1097afbf-4798-4695-8714-8f343b8646fc-installation-pull-secrets\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.402972 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.402817 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1097afbf-4798-4695-8714-8f343b8646fc-bound-sa-token\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.402972 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.402875 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1097afbf-4798-4695-8714-8f343b8646fc-image-registry-private-configuration\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.402972 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.402943 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1097afbf-4798-4695-8714-8f343b8646fc-registry-tls\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.403096 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.402977 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1097afbf-4798-4695-8714-8f343b8646fc-trusted-ca\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.403096 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.402999 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1097afbf-4798-4695-8714-8f343b8646fc-registry-certificates\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.403096 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.403054 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1097afbf-4798-4695-8714-8f343b8646fc-ca-trust-extracted\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.403202 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.403180 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bt8z\" (UniqueName: \"kubernetes.io/projected/1097afbf-4798-4695-8714-8f343b8646fc-kube-api-access-4bt8z\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.462468 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.462431 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" Jul 03 20:33:18.479166 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.479140 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" Jul 03 20:33:18.484039 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.484014 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz" Jul 03 20:33:18.504427 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.504392 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1097afbf-4798-4695-8714-8f343b8646fc-registry-tls\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.504601 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.504442 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1097afbf-4798-4695-8714-8f343b8646fc-trusted-ca\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.504601 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.504474 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1097afbf-4798-4695-8714-8f343b8646fc-registry-certificates\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.504601 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.504501 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1097afbf-4798-4695-8714-8f343b8646fc-ca-trust-extracted\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.504601 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.504536 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bt8z\" (UniqueName: \"kubernetes.io/projected/1097afbf-4798-4695-8714-8f343b8646fc-kube-api-access-4bt8z\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.504601 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.504575 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1097afbf-4798-4695-8714-8f343b8646fc-installation-pull-secrets\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.504601 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.504599 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1097afbf-4798-4695-8714-8f343b8646fc-bound-sa-token\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.504922 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.504647 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1097afbf-4798-4695-8714-8f343b8646fc-image-registry-private-configuration\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.505398 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.505329 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1097afbf-4798-4695-8714-8f343b8646fc-ca-trust-extracted\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.506237 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.506202 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1097afbf-4798-4695-8714-8f343b8646fc-registry-certificates\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.506330 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.506294 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1097afbf-4798-4695-8714-8f343b8646fc-trusted-ca\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.508183 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.508159 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1097afbf-4798-4695-8714-8f343b8646fc-image-registry-private-configuration\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.508293 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.508254 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1097afbf-4798-4695-8714-8f343b8646fc-installation-pull-secrets\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.509099 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.509055 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1097afbf-4798-4695-8714-8f343b8646fc-registry-tls\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.517729 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.517698 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bt8z\" (UniqueName: \"kubernetes.io/projected/1097afbf-4798-4695-8714-8f343b8646fc-kube-api-access-4bt8z\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.518119 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.518094 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1097afbf-4798-4695-8714-8f343b8646fc-bound-sa-token\") pod \"image-registry-684848dfc7-kqrbk\" (UID: \"1097afbf-4798-4695-8714-8f343b8646fc\") " pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.601532 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.601504 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:18.609020 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.608868 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf"] Jul 03 20:33:18.612614 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:33:18.612571 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb5957a1_4bb1_4f2e_8199_c9fecb20a624.slice/crio-809189fc961b2f3d5a7e7846e1593bd874ed5b421fa9238f2dc5199229ac179c WatchSource:0}: Error finding container 809189fc961b2f3d5a7e7846e1593bd874ed5b421fa9238f2dc5199229ac179c: Status 404 returned error can't find the container with id 809189fc961b2f3d5a7e7846e1593bd874ed5b421fa9238f2dc5199229ac179c Jul 03 20:33:18.630480 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.630454 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d"] Jul 03 20:33:18.634033 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:33:18.633997 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46f0ed82_54bb_4065_832f_07146828a892.slice/crio-378934c89823ca505021352275294a9c325e6610ada7b48140606ae3b395a1dc WatchSource:0}: Error finding container 378934c89823ca505021352275294a9c325e6610ada7b48140606ae3b395a1dc: Status 404 returned error can't find the container with id 378934c89823ca505021352275294a9c325e6610ada7b48140606ae3b395a1dc Jul 03 20:33:18.647989 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.647964 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz"] Jul 03 20:33:18.651985 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:33:18.651953 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b72694b_08b9_445f_8129_60b0de44040a.slice/crio-e09900b4693b761fe5001062136fc9b42ae560bc546a96f568e1d486e4420ff8 WatchSource:0}: Error finding container e09900b4693b761fe5001062136fc9b42ae560bc546a96f568e1d486e4420ff8: Status 404 returned error can't find the container with id e09900b4693b761fe5001062136fc9b42ae560bc546a96f568e1d486e4420ff8 Jul 03 20:33:18.736924 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:18.736894 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-684848dfc7-kqrbk"] Jul 03 20:33:18.739495 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:33:18.739467 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1097afbf_4798_4695_8714_8f343b8646fc.slice/crio-7c19125a969382f5a844a72c85369e45d59776b89da1d08912ac2b07c43fb2aa WatchSource:0}: Error finding container 7c19125a969382f5a844a72c85369e45d59776b89da1d08912ac2b07c43fb2aa: Status 404 returned error can't find the container with id 7c19125a969382f5a844a72c85369e45d59776b89da1d08912ac2b07c43fb2aa Jul 03 20:33:19.307495 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.306912 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" event={"ID":"1097afbf-4798-4695-8714-8f343b8646fc","Type":"ContainerStarted","Data":"1bc5e2afa3baa8916079ca8ead6248c55df6209961a947ac0a139b3f709554c9"} Jul 03 20:33:19.307495 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.306973 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" event={"ID":"1097afbf-4798-4695-8714-8f343b8646fc","Type":"ContainerStarted","Data":"7c19125a969382f5a844a72c85369e45d59776b89da1d08912ac2b07c43fb2aa"} Jul 03 20:33:19.307495 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.307231 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:19.310498 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.310428 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" event={"ID":"46f0ed82-54bb-4065-832f-07146828a892","Type":"ContainerStarted","Data":"378934c89823ca505021352275294a9c325e6610ada7b48140606ae3b395a1dc"} Jul 03 20:33:19.314194 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.314166 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz" event={"ID":"6b72694b-08b9-445f-8129-60b0de44040a","Type":"ContainerStarted","Data":"e09900b4693b761fe5001062136fc9b42ae560bc546a96f568e1d486e4420ff8"} Jul 03 20:33:19.316846 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.316819 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" event={"ID":"db5957a1-4bb1-4f2e-8199-c9fecb20a624","Type":"ContainerStarted","Data":"809189fc961b2f3d5a7e7846e1593bd874ed5b421fa9238f2dc5199229ac179c"} Jul 03 20:33:19.333137 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.333086 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" podStartSLOduration=1.333067843 podStartE2EDuration="1.333067843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:33:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:33:19.330958456 +0000 UTC m=+65.789230897" watchObservedRunningTime="2026-07-03 20:33:19.333067843 +0000 UTC m=+65.791340287" Jul 03 20:33:19.717681 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.717543 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:33:19.720452 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.720243 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 03 20:33:19.739418 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.739388 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50e502c5-dd61-4c6f-9bcf-6de839bf5fbb-original-pull-secret\") pod \"global-pull-secret-syncer-chq7m\" (UID: \"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb\") " pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:33:19.818999 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.818281 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5hzv\" (UniqueName: \"kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv\") pod \"network-check-target-vwpt4\" (UID: \"2494ce49-fb0d-4796-9b59-63dfd2d2c7fd\") " pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:33:19.818999 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.818339 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs\") pod \"network-metrics-daemon-s5vq9\" (UID: \"499e1abf-db52-4ca2-9ea6-208a405209f3\") " pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:33:19.821680 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.821297 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 03 20:33:19.821680 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.821527 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 03 20:33:19.831831 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.831630 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 03 20:33:19.835449 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.835426 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/499e1abf-db52-4ca2-9ea6-208a405209f3-metrics-certs\") pod \"network-metrics-daemon-s5vq9\" (UID: \"499e1abf-db52-4ca2-9ea6-208a405209f3\") " pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:33:19.845835 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.845784 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5hzv\" (UniqueName: \"kubernetes.io/projected/2494ce49-fb0d-4796-9b59-63dfd2d2c7fd-kube-api-access-v5hzv\") pod \"network-check-target-vwpt4\" (UID: \"2494ce49-fb0d-4796-9b59-63dfd2d2c7fd\") " pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:33:19.891783 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.891487 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chq7m" Jul 03 20:33:19.905537 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.905279 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9tq75\"" Jul 03 20:33:19.909381 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.909201 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k9k4n\"" Jul 03 20:33:19.913261 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.913076 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s5vq9" Jul 03 20:33:19.918353 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:19.918335 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:33:20.117776 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:20.116797 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-chq7m"] Jul 03 20:33:20.123279 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:33:20.123193 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50e502c5_dd61_4c6f_9bcf_6de839bf5fbb.slice/crio-edef0eb0c4f7aafeecd0f09367183deb0b4d05e45a61a39075d2d1d7634c9e31 WatchSource:0}: Error finding container edef0eb0c4f7aafeecd0f09367183deb0b4d05e45a61a39075d2d1d7634c9e31: Status 404 returned error can't find the container with id edef0eb0c4f7aafeecd0f09367183deb0b4d05e45a61a39075d2d1d7634c9e31 Jul 03 20:33:20.141871 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:20.141795 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s5vq9"] Jul 03 20:33:20.151206 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:33:20.151174 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod499e1abf_db52_4ca2_9ea6_208a405209f3.slice/crio-75b9972d99ff2e24d2f516e487bd737eab50ddacac66b957dafae1e0b7d6e3d8 WatchSource:0}: Error finding container 75b9972d99ff2e24d2f516e487bd737eab50ddacac66b957dafae1e0b7d6e3d8: Status 404 returned error can't find the container with id 75b9972d99ff2e24d2f516e487bd737eab50ddacac66b957dafae1e0b7d6e3d8 Jul 03 20:33:20.166271 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:20.166082 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vwpt4"] Jul 03 20:33:20.168637 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:33:20.168608 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2494ce49_fb0d_4796_9b59_63dfd2d2c7fd.slice/crio-70b2c1781fd130be451796e14790ddd1cf6933fedd92022b1757baec3f407cc4 WatchSource:0}: Error finding container 70b2c1781fd130be451796e14790ddd1cf6933fedd92022b1757baec3f407cc4: Status 404 returned error can't find the container with id 70b2c1781fd130be451796e14790ddd1cf6933fedd92022b1757baec3f407cc4 Jul 03 20:33:20.334025 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:20.333940 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s5vq9" event={"ID":"499e1abf-db52-4ca2-9ea6-208a405209f3","Type":"ContainerStarted","Data":"75b9972d99ff2e24d2f516e487bd737eab50ddacac66b957dafae1e0b7d6e3d8"} Jul 03 20:33:20.340728 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:20.340654 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-chq7m" event={"ID":"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb","Type":"ContainerStarted","Data":"edef0eb0c4f7aafeecd0f09367183deb0b4d05e45a61a39075d2d1d7634c9e31"} Jul 03 20:33:20.344216 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:20.344155 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vwpt4" event={"ID":"2494ce49-fb0d-4796-9b59-63dfd2d2c7fd","Type":"ContainerStarted","Data":"70b2c1781fd130be451796e14790ddd1cf6933fedd92022b1757baec3f407cc4"} Jul 03 20:33:26.474269 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:26.474222 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:33:26.477116 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:26.477075 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2004c927-d12e-4dc3-9c28-01d6e488fc0e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j67\" (UID: \"2004c927-d12e-4dc3-9c28-01d6e488fc0e\") " pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:33:26.654893 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:26.654863 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-498s6\"" Jul 03 20:33:26.662274 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:26.662250 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v7j67" Jul 03 20:33:27.996987 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:27.995949 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v7j67"] Jul 03 20:33:28.146879 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.146637 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:33:28.377340 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.377256 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" event={"ID":"46f0ed82-54bb-4065-832f-07146828a892","Type":"ContainerStarted","Data":"8564b5bd5de1f4d22f4e1d87fe6d8a223417aa827cc8306de35ff6df0d5850f2"} Jul 03 20:33:28.377340 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.377306 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" event={"ID":"46f0ed82-54bb-4065-832f-07146828a892","Type":"ContainerStarted","Data":"21e57fa86edd6ed386feb62433791f02b23db0efcd10a3be59791cb9952ae2c4"} Jul 03 20:33:28.377340 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.377321 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" event={"ID":"46f0ed82-54bb-4065-832f-07146828a892","Type":"ContainerStarted","Data":"5cb781d39d5a342d507df472dd9007e2561f1ed053aaadf375ab8dda0f52a9b5"} Jul 03 20:33:28.378922 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.378898 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s5vq9" event={"ID":"499e1abf-db52-4ca2-9ea6-208a405209f3","Type":"ContainerStarted","Data":"24488bbfb95ec038d48d12a3ed96e6f82e9e485fc0e48a016e9e6b2ec4e4aced"} Jul 03 20:33:28.379041 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.378948 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s5vq9" event={"ID":"499e1abf-db52-4ca2-9ea6-208a405209f3","Type":"ContainerStarted","Data":"e37bb11bccf04c87290b7f75a7e7a893237a08d1b8b1e753e87fe5511125dee4"} Jul 03 20:33:28.380329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.380304 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-chq7m" event={"ID":"50e502c5-dd61-4c6f-9bcf-6de839bf5fbb","Type":"ContainerStarted","Data":"f20fa2074e4926484dc052934aea033d7b55e42905dee335acf52f1002a71e5a"} Jul 03 20:33:28.381893 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.381872 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz" event={"ID":"6b72694b-08b9-445f-8129-60b0de44040a","Type":"ContainerStarted","Data":"e45960125b68fcba8628bd51d0dc07614df378ac2364be254b8d674cbe447c97"} Jul 03 20:33:28.383389 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.383362 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" event={"ID":"db5957a1-4bb1-4f2e-8199-c9fecb20a624","Type":"ContainerStarted","Data":"f74d5cb8aadf1a21d31d127616d2b228ec9b13b114c4ce422922d40e8de0adec"} Jul 03 20:33:28.383524 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.383491 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" Jul 03 20:33:28.384996 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.384970 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vwpt4" event={"ID":"2494ce49-fb0d-4796-9b59-63dfd2d2c7fd","Type":"ContainerStarted","Data":"03baca58b74f5dec56093a7cd6af78fe6dfd5f38c1e5a34c55c3f2ec4e139640"} Jul 03 20:33:28.385109 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.385025 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:33:28.385231 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.385216 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" Jul 03 20:33:28.386194 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.386174 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v7j67" event={"ID":"2004c927-d12e-4dc3-9c28-01d6e488fc0e","Type":"ContainerStarted","Data":"3fe38c456193b6c5f175b27c8a7b6690e1a57536c461c5f461cfbe29947e6daf"} Jul 03 20:33:28.406204 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.406140 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b6c8d67fc-sqw2d" podStartSLOduration=1.223695208 podStartE2EDuration="10.406121969s" totalImagesPullingTime="9.182426761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:33:28.405388712 +0000 UTC m=+74.863661152" watchObservedRunningTime="2026-07-03 20:33:28.406121969 +0000 UTC m=+74.864394409" Jul 03 20:33:28.443259 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.443198 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s5vq9" podStartSLOduration=66.735715665 podStartE2EDuration="1m14.443178075s" totalImagesPullingTime="7.70746241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:33:28.424018726 +0000 UTC m=+74.882291166" watchObservedRunningTime="2026-07-03 20:33:28.443178075 +0000 UTC m=+74.901450515" Jul 03 20:33:28.460059 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.459692 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vwpt4" podStartSLOduration=66.778094357 podStartE2EDuration="1m14.459673355s" totalImagesPullingTime="7.681578998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:33:28.442805518 +0000 UTC m=+74.901077951" watchObservedRunningTime="2026-07-03 20:33:28.459673355 +0000 UTC m=+74.917945795" Jul 03 20:33:28.478287 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.476818 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787dbf8ddb-lk5mz" podStartSLOduration=1.5365046329999998 podStartE2EDuration="10.47680041s" totalImagesPullingTime="8.940295777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:33:28.458445556 +0000 UTC m=+74.916717990" watchObservedRunningTime="2026-07-03 20:33:28.47680041 +0000 UTC m=+74.935072851" Jul 03 20:33:28.496451 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.494999 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-chq7m" podStartSLOduration=66.772650426 podStartE2EDuration="1m14.494981597s" totalImagesPullingTime="7.722331171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:33:28.494944909 +0000 UTC m=+74.953217346" watchObservedRunningTime="2026-07-03 20:33:28.494981597 +0000 UTC m=+74.953254029" Jul 03 20:33:28.496451 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:28.495530 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77b6ffb954-8gkvf" podStartSLOduration=1.2859762350000001 podStartE2EDuration="10.495515997s" totalImagesPullingTime="9.209539762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:33:28.478381033 +0000 UTC m=+74.936653473" watchObservedRunningTime="2026-07-03 20:33:28.495515997 +0000 UTC m=+74.953788437" Jul 03 20:33:31.897776 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:31.896205 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qhn7n"] Jul 03 20:33:31.901141 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:31.900607 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:31.906359 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:31.905105 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 03 20:33:31.906359 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:31.905312 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 03 20:33:31.906359 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:31.905470 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 03 20:33:31.906359 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:31.905617 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 03 20:33:31.906359 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:31.905779 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 03 20:33:31.906359 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:31.906038 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 03 20:33:31.906359 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:31.906225 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dgnqf\"" Jul 03 20:33:32.028181 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.028090 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-sys\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.028181 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.028130 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-wtmp\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.028181 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.028151 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-accelerators-collector-config\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.028455 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.028265 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-root\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.028455 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.028301 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-textfile\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.028455 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.028325 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-metrics-client-ca\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.028455 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.028383 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-tls\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.028455 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.028434 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc6dc\" (UniqueName: \"kubernetes.io/projected/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-kube-api-access-gc6dc\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.028642 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.028468 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.129841 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.129805 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc6dc\" (UniqueName: \"kubernetes.io/projected/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-kube-api-access-gc6dc\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.129841 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.129845 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.130083 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.129865 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-sys\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.130083 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.129882 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-wtmp\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.130083 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.129900 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-accelerators-collector-config\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.130083 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.129945 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-root\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.130083 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.129964 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-sys\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.130083 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.129971 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-textfile\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.130083 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.130005 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-metrics-client-ca\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.130083 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.130044 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-tls\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.130083 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.130071 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-wtmp\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.130463 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:33:32.130135 2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 03 20:33:32.130463 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:33:32.130191 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-tls podName:be7d78b0-7c53-41cf-805e-65c97e3b0ec2 nodeName:}" failed. No retries permitted until 2026-07-03 20:33:32.630170737 +0000 UTC m=+79.088443155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-tls") pod "node-exporter-qhn7n" (UID: "be7d78b0-7c53-41cf-805e-65c97e3b0ec2") : secret "node-exporter-tls" not found Jul 03 20:33:32.130463 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.130225 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-root\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.130463 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.130378 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-textfile\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.130631 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.130608 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-accelerators-collector-config\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.130870 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.130854 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-metrics-client-ca\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.132522 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.132502 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.140908 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.140878 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc6dc\" (UniqueName: \"kubernetes.io/projected/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-kube-api-access-gc6dc\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.405265 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.405225 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v7j67" event={"ID":"2004c927-d12e-4dc3-9c28-01d6e488fc0e","Type":"ContainerStarted","Data":"9c169f10636cb72a4ed20d7bc57c5dca2b0fb4563269d7217f35aaa1e43ede67"} Jul 03 20:33:32.634107 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.634077 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-tls\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.636642 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.636616 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/be7d78b0-7c53-41cf-805e-65c97e3b0ec2-node-exporter-tls\") pod \"node-exporter-qhn7n\" (UID: \"be7d78b0-7c53-41cf-805e-65c97e3b0ec2\") " pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.817197 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:32.817101 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qhn7n" Jul 03 20:33:32.828426 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:33:32.828402 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe7d78b0_7c53_41cf_805e_65c97e3b0ec2.slice/crio-a6e144f9f5ab80f4ba54a35478ef1b9d61ec91784cb4926417d0824e7c914982 WatchSource:0}: Error finding container a6e144f9f5ab80f4ba54a35478ef1b9d61ec91784cb4926417d0824e7c914982: Status 404 returned error can't find the container with id a6e144f9f5ab80f4ba54a35478ef1b9d61ec91784cb4926417d0824e7c914982 Jul 03 20:33:33.415776 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:33.413334 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v7j67" event={"ID":"2004c927-d12e-4dc3-9c28-01d6e488fc0e","Type":"ContainerStarted","Data":"677f2fd36d30bc3fce8581a9402b989ecd6e242e18b14a0290c5b726272a08c5"} Jul 03 20:33:33.420653 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:33.420611 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qhn7n" event={"ID":"be7d78b0-7c53-41cf-805e-65c97e3b0ec2","Type":"ContainerStarted","Data":"a6e144f9f5ab80f4ba54a35478ef1b9d61ec91784cb4926417d0824e7c914982"} Jul 03 20:33:34.426738 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:34.426603 2582 generic.go:356] "Generic (PLEG): container finished" podID="be7d78b0-7c53-41cf-805e-65c97e3b0ec2" containerID="ed1ad141b9a4614ffcf74cc432b701e3c58055e136f7010d699662073e3b9984" exitCode=0 Jul 03 20:33:34.426738 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:34.426684 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qhn7n" event={"ID":"be7d78b0-7c53-41cf-805e-65c97e3b0ec2","Type":"ContainerDied","Data":"ed1ad141b9a4614ffcf74cc432b701e3c58055e136f7010d699662073e3b9984"} Jul 03 20:33:35.432647 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:35.432607 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v7j67" event={"ID":"2004c927-d12e-4dc3-9c28-01d6e488fc0e","Type":"ContainerStarted","Data":"f3d3599ee4684d00c58a23ce109e4db24d6feafaab89ceb40dcb275991921aa3"} Jul 03 20:33:35.434856 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:35.434832 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qhn7n" event={"ID":"be7d78b0-7c53-41cf-805e-65c97e3b0ec2","Type":"ContainerStarted","Data":"468d83990fe6b2789e002f23f72bdca698c9325a85225a98de2934627b7f9433"} Jul 03 20:33:35.434966 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:35.434877 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qhn7n" event={"ID":"be7d78b0-7c53-41cf-805e-65c97e3b0ec2","Type":"ContainerStarted","Data":"2e0298a4115469810c4a18d97f5bd34f516ee769efa59ff31b2a026e9a07dca4"} Jul 03 20:33:35.450309 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:35.450270 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v7j67" podStartSLOduration=34.55926938 podStartE2EDuration="41.450256528s" totalImagesPullingTime="6.890987148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:32:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:33:35.448945706 +0000 UTC m=+81.907218145" watchObservedRunningTime="2026-07-03 20:33:35.450256528 +0000 UTC m=+81.908528966" Jul 03 20:33:35.465411 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:35.465371 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qhn7n" podStartSLOduration=3.491302855 podStartE2EDuration="4.465359s" totalImagesPullingTime="895.0374ms" totalInitContainerRuntime="79.018745ms" isStatefulPod=true podCreationTimestamp="2026-07-03 20:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:33:35.464805126 +0000 UTC m=+81.923077564" watchObservedRunningTime="2026-07-03 20:33:35.465359 +0000 UTC m=+81.923631439" Jul 03 20:33:40.349222 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:40.349191 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-684848dfc7-kqrbk" Jul 03 20:33:43.163295 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.163232 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" podUID="31eaf79b-1cf2-4dcd-8872-d43d5701d9f3" containerName="registry" containerID="cri-o://632d96d11139e6daa6d9994b853a9877c7939e2670341d2ec47b060116795f2d" gracePeriod=30 Jul 03 20:33:43.406072 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.406053 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:33:43.417435 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.417376 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-ca-trust-extracted\") pod \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " Jul 03 20:33:43.417435 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.417409 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls\") pod \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " Jul 03 20:33:43.417435 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.417428 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-image-registry-private-configuration\") pod \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " Jul 03 20:33:43.417646 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.417445 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-certificates\") pod \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " Jul 03 20:33:43.417646 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.417468 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-bound-sa-token\") pod \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " Jul 03 20:33:43.417646 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.417502 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-installation-pull-secrets\") pod \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " Jul 03 20:33:43.417646 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.417532 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8mrk\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-kube-api-access-k8mrk\") pod \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " Jul 03 20:33:43.417646 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.417558 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-trusted-ca\") pod \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\" (UID: \"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3\") " Jul 03 20:33:43.418110 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.417887 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3" (UID: "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:33:43.418187 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.418145 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3" (UID: "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:33:43.420230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.420192 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3" (UID: "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:33:43.420341 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.420282 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3" (UID: "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:33:43.420341 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.420305 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3" (UID: "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:33:43.420447 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.420365 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-kube-api-access-k8mrk" (OuterVolumeSpecName: "kube-api-access-k8mrk") pod "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3" (UID: "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3"). InnerVolumeSpecName "kube-api-access-k8mrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:33:43.420520 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.420500 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3" (UID: "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:33:43.426869 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.426839 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3" (UID: "31eaf79b-1cf2-4dcd-8872-d43d5701d9f3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:33:43.462568 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.462538 2582 generic.go:356] "Generic (PLEG): container finished" podID="31eaf79b-1cf2-4dcd-8872-d43d5701d9f3" containerID="632d96d11139e6daa6d9994b853a9877c7939e2670341d2ec47b060116795f2d" exitCode=0 Jul 03 20:33:43.462662 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.462581 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" Jul 03 20:33:43.462662 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.462617 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" event={"ID":"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3","Type":"ContainerDied","Data":"632d96d11139e6daa6d9994b853a9877c7939e2670341d2ec47b060116795f2d"} Jul 03 20:33:43.462735 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.462691 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d798d84bc-2plw8" event={"ID":"31eaf79b-1cf2-4dcd-8872-d43d5701d9f3","Type":"ContainerDied","Data":"bbd173253d587c426dc346fb24c56a6cf3a9f3afc779e3af9111260614075cbd"} Jul 03 20:33:43.462735 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.462711 2582 scope.go:117] "RemoveContainer" containerID="632d96d11139e6daa6d9994b853a9877c7939e2670341d2ec47b060116795f2d" Jul 03 20:33:43.471042 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.471024 2582 scope.go:117] "RemoveContainer" containerID="632d96d11139e6daa6d9994b853a9877c7939e2670341d2ec47b060116795f2d" Jul 03 20:33:43.471283 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:33:43.471262 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"632d96d11139e6daa6d9994b853a9877c7939e2670341d2ec47b060116795f2d\": container with ID starting with 632d96d11139e6daa6d9994b853a9877c7939e2670341d2ec47b060116795f2d not found: ID does not exist" containerID="632d96d11139e6daa6d9994b853a9877c7939e2670341d2ec47b060116795f2d" Jul 03 20:33:43.471354 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.471295 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"632d96d11139e6daa6d9994b853a9877c7939e2670341d2ec47b060116795f2d"} err="failed to get container status \"632d96d11139e6daa6d9994b853a9877c7939e2670341d2ec47b060116795f2d\": rpc error: code = NotFound desc = could not find container \"632d96d11139e6daa6d9994b853a9877c7939e2670341d2ec47b060116795f2d\": container with ID starting with 632d96d11139e6daa6d9994b853a9877c7939e2670341d2ec47b060116795f2d not found: ID does not exist" Jul 03 20:33:43.481604 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.481586 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d798d84bc-2plw8"] Jul 03 20:33:43.485459 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.485439 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5d798d84bc-2plw8"] Jul 03 20:33:43.518339 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.518314 2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-installation-pull-secrets\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:33:43.518339 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.518337 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8mrk\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-kube-api-access-k8mrk\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:33:43.518484 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.518353 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-trusted-ca\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:33:43.518484 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.518365 2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-ca-trust-extracted\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:33:43.518484 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.518376 2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-tls\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:33:43.518484 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.518388 2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-image-registry-private-configuration\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:33:43.518484 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.518401 2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-registry-certificates\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:33:43.518484 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.518414 2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3-bound-sa-token\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:33:43.984871 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:43.984835 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31eaf79b-1cf2-4dcd-8872-d43d5701d9f3" path="/var/lib/kubelet/pods/31eaf79b-1cf2-4dcd-8872-d43d5701d9f3/volumes" Jul 03 20:33:59.394202 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:33:59.394169 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vwpt4" Jul 03 20:37:13.883333 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:13.883305 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 03 20:37:45.594632 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.594599 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh"] Jul 03 20:37:45.596482 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.594871 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31eaf79b-1cf2-4dcd-8872-d43d5701d9f3" containerName="registry" Jul 03 20:37:45.596482 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.594882 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="31eaf79b-1cf2-4dcd-8872-d43d5701d9f3" containerName="registry" Jul 03 20:37:45.596482 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.594930 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="31eaf79b-1cf2-4dcd-8872-d43d5701d9f3" containerName="registry" Jul 03 20:37:45.596876 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.596859 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.600893 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.600871 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 03 20:37:45.603323 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.603065 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 03 20:37:45.603323 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.603159 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-p699g\"" Jul 03 20:37:45.603323 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.603232 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 03 20:37:45.603953 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.603937 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 03 20:37:45.652693 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.652668 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b1a081df-9d43-4d9b-a005-df7b6639a7e9-manifests\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.652826 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.652701 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b1a081df-9d43-4d9b-a005-df7b6639a7e9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.652826 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.652730 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1a081df-9d43-4d9b-a005-df7b6639a7e9-tmp\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.652826 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.652806 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1a081df-9d43-4d9b-a005-df7b6639a7e9-webhook-cert\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.652955 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.652879 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gg8h\" (UniqueName: \"kubernetes.io/projected/b1a081df-9d43-4d9b-a005-df7b6639a7e9-kube-api-access-5gg8h\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.714290 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.714259 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh"] Jul 03 20:37:45.753337 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.753311 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1a081df-9d43-4d9b-a005-df7b6639a7e9-tmp\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.753493 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.753341 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1a081df-9d43-4d9b-a005-df7b6639a7e9-webhook-cert\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.753493 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.753367 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gg8h\" (UniqueName: \"kubernetes.io/projected/b1a081df-9d43-4d9b-a005-df7b6639a7e9-kube-api-access-5gg8h\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.753493 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.753416 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b1a081df-9d43-4d9b-a005-df7b6639a7e9-manifests\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.753632 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.753534 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b1a081df-9d43-4d9b-a005-df7b6639a7e9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.753880 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.753854 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b1a081df-9d43-4d9b-a005-df7b6639a7e9-manifests\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.755838 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.755813 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1a081df-9d43-4d9b-a005-df7b6639a7e9-tmp\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.755958 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.755942 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1a081df-9d43-4d9b-a005-df7b6639a7e9-webhook-cert\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.756000 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.755987 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b1a081df-9d43-4d9b-a005-df7b6639a7e9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.764384 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.764359 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gg8h\" (UniqueName: \"kubernetes.io/projected/b1a081df-9d43-4d9b-a005-df7b6639a7e9-kube-api-access-5gg8h\") pod \"opendatahub-operator-controller-manager-8554cfd646-8l6wh\" (UID: \"b1a081df-9d43-4d9b-a005-df7b6639a7e9\") " pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:45.906624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:45.906592 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:46.036554 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:46.036368 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh"] Jul 03 20:37:46.039290 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:37:46.039263 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1a081df_9d43_4d9b_a005_df7b6639a7e9.slice/crio-40ca82d374345970ab499c45aa51060d145a52af67355f4f768ac4940ff6c465 WatchSource:0}: Error finding container 40ca82d374345970ab499c45aa51060d145a52af67355f4f768ac4940ff6c465: Status 404 returned error can't find the container with id 40ca82d374345970ab499c45aa51060d145a52af67355f4f768ac4940ff6c465 Jul 03 20:37:46.041121 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:46.041106 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 20:37:46.256986 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:46.256898 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" event={"ID":"b1a081df-9d43-4d9b-a005-df7b6639a7e9","Type":"ContainerStarted","Data":"40ca82d374345970ab499c45aa51060d145a52af67355f4f768ac4940ff6c465"} Jul 03 20:37:49.275870 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:49.275834 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" event={"ID":"b1a081df-9d43-4d9b-a005-df7b6639a7e9","Type":"ContainerStarted","Data":"50f4afa44f24ede6a59a14d08d4ae0aabc653be321c9f54572fa73550cc41ef4"} Jul 03 20:37:50.280814 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:50.280725 2582 generic.go:356] "Generic (PLEG): container finished" podID="b1a081df-9d43-4d9b-a005-df7b6639a7e9" containerID="50f4afa44f24ede6a59a14d08d4ae0aabc653be321c9f54572fa73550cc41ef4" exitCode=0 Jul 03 20:37:50.281170 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:50.280812 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" event={"ID":"b1a081df-9d43-4d9b-a005-df7b6639a7e9","Type":"ContainerDied","Data":"50f4afa44f24ede6a59a14d08d4ae0aabc653be321c9f54572fa73550cc41ef4"} Jul 03 20:37:51.287097 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:51.287060 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" event={"ID":"b1a081df-9d43-4d9b-a005-df7b6639a7e9","Type":"ContainerStarted","Data":"b0114b49fee8d08b376ba4e7b239d9d3712bcb569867c7655231020cce86a17c"} Jul 03 20:37:51.287471 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:51.287142 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:37:51.309129 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:37:51.309077 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" podStartSLOduration=2.07255344 podStartE2EDuration="6.309061127s" totalImagesPullingTime="3.182531333s" totalInitContainerRuntime="1.053976354s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:37:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:37:51.307068166 +0000 UTC m=+337.765340607" watchObservedRunningTime="2026-07-03 20:37:51.309061127 +0000 UTC m=+337.767333566" Jul 03 20:38:02.293497 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:02.293463 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8554cfd646-8l6wh" Jul 03 20:38:05.400089 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.399996 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5546b876fd-gqk58"] Jul 03 20:38:05.403385 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.403367 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5546b876fd-gqk58" Jul 03 20:38:05.407480 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.407457 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 03 20:38:05.407581 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.407479 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 03 20:38:05.408003 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.407984 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 03 20:38:05.408143 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.408126 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-b4ft4\"" Jul 03 20:38:05.408393 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.408378 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 03 20:38:05.439937 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.439915 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5546b876fd-gqk58"] Jul 03 20:38:05.501609 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.501582 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a2d0db1-3f77-4931-b456-58ea5a1f46de-tls-certs\") pod \"kube-auth-proxy-5546b876fd-gqk58\" (UID: \"4a2d0db1-3f77-4931-b456-58ea5a1f46de\") " pod="openshift-ingress/kube-auth-proxy-5546b876fd-gqk58" Jul 03 20:38:05.501765 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.501644 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a2d0db1-3f77-4931-b456-58ea5a1f46de-tmp\") pod \"kube-auth-proxy-5546b876fd-gqk58\" (UID: \"4a2d0db1-3f77-4931-b456-58ea5a1f46de\") " pod="openshift-ingress/kube-auth-proxy-5546b876fd-gqk58" Jul 03 20:38:05.501765 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.501699 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2shzx\" (UniqueName: \"kubernetes.io/projected/4a2d0db1-3f77-4931-b456-58ea5a1f46de-kube-api-access-2shzx\") pod \"kube-auth-proxy-5546b876fd-gqk58\" (UID: \"4a2d0db1-3f77-4931-b456-58ea5a1f46de\") " pod="openshift-ingress/kube-auth-proxy-5546b876fd-gqk58" Jul 03 20:38:05.602808 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.602775 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a2d0db1-3f77-4931-b456-58ea5a1f46de-tmp\") pod \"kube-auth-proxy-5546b876fd-gqk58\" (UID: \"4a2d0db1-3f77-4931-b456-58ea5a1f46de\") " pod="openshift-ingress/kube-auth-proxy-5546b876fd-gqk58" Jul 03 20:38:05.602995 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.602820 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2shzx\" (UniqueName: \"kubernetes.io/projected/4a2d0db1-3f77-4931-b456-58ea5a1f46de-kube-api-access-2shzx\") pod \"kube-auth-proxy-5546b876fd-gqk58\" (UID: \"4a2d0db1-3f77-4931-b456-58ea5a1f46de\") " pod="openshift-ingress/kube-auth-proxy-5546b876fd-gqk58" Jul 03 20:38:05.602995 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.602862 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a2d0db1-3f77-4931-b456-58ea5a1f46de-tls-certs\") pod \"kube-auth-proxy-5546b876fd-gqk58\" (UID: \"4a2d0db1-3f77-4931-b456-58ea5a1f46de\") " pod="openshift-ingress/kube-auth-proxy-5546b876fd-gqk58" Jul 03 20:38:05.605144 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.605116 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a2d0db1-3f77-4931-b456-58ea5a1f46de-tmp\") pod \"kube-auth-proxy-5546b876fd-gqk58\" (UID: \"4a2d0db1-3f77-4931-b456-58ea5a1f46de\") " pod="openshift-ingress/kube-auth-proxy-5546b876fd-gqk58" Jul 03 20:38:05.605367 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.605349 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a2d0db1-3f77-4931-b456-58ea5a1f46de-tls-certs\") pod \"kube-auth-proxy-5546b876fd-gqk58\" (UID: \"4a2d0db1-3f77-4931-b456-58ea5a1f46de\") " pod="openshift-ingress/kube-auth-proxy-5546b876fd-gqk58" Jul 03 20:38:05.631587 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.631562 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2shzx\" (UniqueName: \"kubernetes.io/projected/4a2d0db1-3f77-4931-b456-58ea5a1f46de-kube-api-access-2shzx\") pod \"kube-auth-proxy-5546b876fd-gqk58\" (UID: \"4a2d0db1-3f77-4931-b456-58ea5a1f46de\") " pod="openshift-ingress/kube-auth-proxy-5546b876fd-gqk58" Jul 03 20:38:05.712684 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.712621 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5546b876fd-gqk58" Jul 03 20:38:05.865873 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.865841 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5546b876fd-gqk58"] Jul 03 20:38:05.869096 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:38:05.869052 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a2d0db1_3f77_4931_b456_58ea5a1f46de.slice/crio-68a54f46e955038a9536da964bb5f54a8adbf67103e25606706d30921dceac26 WatchSource:0}: Error finding container 68a54f46e955038a9536da964bb5f54a8adbf67103e25606706d30921dceac26: Status 404 returned error can't find the container with id 68a54f46e955038a9536da964bb5f54a8adbf67103e25606706d30921dceac26 Jul 03 20:38:05.911698 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.911659 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6tfkc"] Jul 03 20:38:05.915650 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.915625 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6tfkc" Jul 03 20:38:05.923102 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.923077 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zcrlv\"" Jul 03 20:38:05.923540 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.923520 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 03 20:38:05.945940 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:05.945913 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6tfkc"] Jul 03 20:38:06.005492 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:06.005405 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91a7f796-addb-4293-9e54-07cd9433cce3-cert\") pod \"odh-model-controller-7998c67dc4-6tfkc\" (UID: \"91a7f796-addb-4293-9e54-07cd9433cce3\") " pod="opendatahub/odh-model-controller-7998c67dc4-6tfkc" Jul 03 20:38:06.005492 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:06.005462 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdnfw\" (UniqueName: \"kubernetes.io/projected/91a7f796-addb-4293-9e54-07cd9433cce3-kube-api-access-tdnfw\") pod \"odh-model-controller-7998c67dc4-6tfkc\" (UID: \"91a7f796-addb-4293-9e54-07cd9433cce3\") " pod="opendatahub/odh-model-controller-7998c67dc4-6tfkc" Jul 03 20:38:06.106011 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:06.105971 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91a7f796-addb-4293-9e54-07cd9433cce3-cert\") pod \"odh-model-controller-7998c67dc4-6tfkc\" (UID: \"91a7f796-addb-4293-9e54-07cd9433cce3\") " pod="opendatahub/odh-model-controller-7998c67dc4-6tfkc" Jul 03 20:38:06.106190 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:06.106023 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdnfw\" (UniqueName: \"kubernetes.io/projected/91a7f796-addb-4293-9e54-07cd9433cce3-kube-api-access-tdnfw\") pod \"odh-model-controller-7998c67dc4-6tfkc\" (UID: \"91a7f796-addb-4293-9e54-07cd9433cce3\") " pod="opendatahub/odh-model-controller-7998c67dc4-6tfkc" Jul 03 20:38:06.106190 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:38:06.106124 2582 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 03 20:38:06.106190 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:38:06.106189 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a7f796-addb-4293-9e54-07cd9433cce3-cert podName:91a7f796-addb-4293-9e54-07cd9433cce3 nodeName:}" failed. No retries permitted until 2026-07-03 20:38:06.606171886 +0000 UTC m=+353.064444326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/91a7f796-addb-4293-9e54-07cd9433cce3-cert") pod "odh-model-controller-7998c67dc4-6tfkc" (UID: "91a7f796-addb-4293-9e54-07cd9433cce3") : secret "odh-model-controller-webhook-cert" not found Jul 03 20:38:06.137207 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:06.137178 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdnfw\" (UniqueName: \"kubernetes.io/projected/91a7f796-addb-4293-9e54-07cd9433cce3-kube-api-access-tdnfw\") pod \"odh-model-controller-7998c67dc4-6tfkc\" (UID: \"91a7f796-addb-4293-9e54-07cd9433cce3\") " pod="opendatahub/odh-model-controller-7998c67dc4-6tfkc" Jul 03 20:38:06.339940 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:06.339849 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5546b876fd-gqk58" event={"ID":"4a2d0db1-3f77-4931-b456-58ea5a1f46de","Type":"ContainerStarted","Data":"68a54f46e955038a9536da964bb5f54a8adbf67103e25606706d30921dceac26"} Jul 03 20:38:06.611893 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:06.611645 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91a7f796-addb-4293-9e54-07cd9433cce3-cert\") pod \"odh-model-controller-7998c67dc4-6tfkc\" (UID: \"91a7f796-addb-4293-9e54-07cd9433cce3\") " pod="opendatahub/odh-model-controller-7998c67dc4-6tfkc" Jul 03 20:38:06.615164 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:06.615132 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91a7f796-addb-4293-9e54-07cd9433cce3-cert\") pod \"odh-model-controller-7998c67dc4-6tfkc\" (UID: \"91a7f796-addb-4293-9e54-07cd9433cce3\") " pod="opendatahub/odh-model-controller-7998c67dc4-6tfkc" Jul 03 20:38:06.825778 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:06.825697 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6tfkc" Jul 03 20:38:07.014905 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:07.014863 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6tfkc"] Jul 03 20:38:07.019328 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:38:07.019292 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a7f796_addb_4293_9e54_07cd9433cce3.slice/crio-8e78a25cff4655159479e50c54a53561d577ad4f5f8cbdc3bc5596537ee6475e WatchSource:0}: Error finding container 8e78a25cff4655159479e50c54a53561d577ad4f5f8cbdc3bc5596537ee6475e: Status 404 returned error can't find the container with id 8e78a25cff4655159479e50c54a53561d577ad4f5f8cbdc3bc5596537ee6475e Jul 03 20:38:07.344874 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:07.344836 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6tfkc" event={"ID":"91a7f796-addb-4293-9e54-07cd9433cce3","Type":"ContainerStarted","Data":"8e78a25cff4655159479e50c54a53561d577ad4f5f8cbdc3bc5596537ee6475e"} Jul 03 20:38:07.800309 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:07.799968 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wd9c5"] Jul 03 20:38:07.808996 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:07.807108 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-wd9c5" Jul 03 20:38:07.812240 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:07.811958 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 03 20:38:07.812240 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:07.812187 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-lf7mk\"" Jul 03 20:38:07.832230 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:07.832108 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wd9c5"] Jul 03 20:38:07.921340 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:07.921302 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ed9b90-a468-468c-be69-e7e41de57781-tls-certs\") pod \"model-serving-api-54d89bd5fc-wd9c5\" (UID: \"e0ed9b90-a468-468c-be69-e7e41de57781\") " pod="opendatahub/model-serving-api-54d89bd5fc-wd9c5" Jul 03 20:38:07.921526 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:07.921455 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcm2r\" (UniqueName: \"kubernetes.io/projected/e0ed9b90-a468-468c-be69-e7e41de57781-kube-api-access-bcm2r\") pod \"model-serving-api-54d89bd5fc-wd9c5\" (UID: \"e0ed9b90-a468-468c-be69-e7e41de57781\") " pod="opendatahub/model-serving-api-54d89bd5fc-wd9c5" Jul 03 20:38:08.023438 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.022794 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcm2r\" (UniqueName: \"kubernetes.io/projected/e0ed9b90-a468-468c-be69-e7e41de57781-kube-api-access-bcm2r\") pod \"model-serving-api-54d89bd5fc-wd9c5\" (UID: \"e0ed9b90-a468-468c-be69-e7e41de57781\") " pod="opendatahub/model-serving-api-54d89bd5fc-wd9c5" Jul 03 20:38:08.023438 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.022876 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ed9b90-a468-468c-be69-e7e41de57781-tls-certs\") pod \"model-serving-api-54d89bd5fc-wd9c5\" (UID: \"e0ed9b90-a468-468c-be69-e7e41de57781\") " pod="opendatahub/model-serving-api-54d89bd5fc-wd9c5" Jul 03 20:38:08.026544 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.026492 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ed9b90-a468-468c-be69-e7e41de57781-tls-certs\") pod \"model-serving-api-54d89bd5fc-wd9c5\" (UID: \"e0ed9b90-a468-468c-be69-e7e41de57781\") " pod="opendatahub/model-serving-api-54d89bd5fc-wd9c5" Jul 03 20:38:08.058187 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.058115 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcm2r\" (UniqueName: \"kubernetes.io/projected/e0ed9b90-a468-468c-be69-e7e41de57781-kube-api-access-bcm2r\") pod \"model-serving-api-54d89bd5fc-wd9c5\" (UID: \"e0ed9b90-a468-468c-be69-e7e41de57781\") " pod="opendatahub/model-serving-api-54d89bd5fc-wd9c5" Jul 03 20:38:08.124218 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.123914 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-wd9c5" Jul 03 20:38:08.471234 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.471200 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-p96xl"] Jul 03 20:38:08.474616 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.474595 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.478521 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.478359 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 03 20:38:08.478521 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.478381 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 03 20:38:08.478736 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.478687 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dxmcx\"" Jul 03 20:38:08.478968 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.478948 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 03 20:38:08.479070 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.479052 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 03 20:38:08.508907 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.508870 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-p96xl"] Jul 03 20:38:08.527429 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.527401 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/47b625e8-c048-4008-8ef5-c94c73f6c5a3-local-certs\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.527600 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.527444 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hz7v\" (UniqueName: \"kubernetes.io/projected/47b625e8-c048-4008-8ef5-c94c73f6c5a3-kube-api-access-8hz7v\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.527600 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.527481 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-token\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.527600 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.527526 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.527600 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.527588 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-cacerts\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.527810 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.527611 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.527810 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.527643 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.629014 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.628967 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/47b625e8-c048-4008-8ef5-c94c73f6c5a3-local-certs\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.629255 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.629025 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hz7v\" (UniqueName: \"kubernetes.io/projected/47b625e8-c048-4008-8ef5-c94c73f6c5a3-kube-api-access-8hz7v\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.629255 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.629056 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-token\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.629255 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.629101 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.630655 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.630493 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-cacerts\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.630655 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.630529 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.630655 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.630563 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.632079 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.632025 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.633680 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.633632 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/47b625e8-c048-4008-8ef5-c94c73f6c5a3-local-certs\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.635586 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.635544 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.641649 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.641628 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.641788 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.641736 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-cacerts\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.649759 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.649700 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-token\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.651782 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.651720 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hz7v\" (UniqueName: \"kubernetes.io/projected/47b625e8-c048-4008-8ef5-c94c73f6c5a3-kube-api-access-8hz7v\") pod \"istiod-openshift-gateway-7574459656-p96xl\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:08.787693 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:08.787601 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:10.187789 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:10.187763 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wd9c5"] Jul 03 20:38:10.189909 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:38:10.189879 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0ed9b90_a468_468c_be69_e7e41de57781.slice/crio-8ecb25ee4b65da8b7e8c7e669c5f5f7f0036b6c829bf41ed3808e76b11168293 WatchSource:0}: Error finding container 8ecb25ee4b65da8b7e8c7e669c5f5f7f0036b6c829bf41ed3808e76b11168293: Status 404 returned error can't find the container with id 8ecb25ee4b65da8b7e8c7e669c5f5f7f0036b6c829bf41ed3808e76b11168293 Jul 03 20:38:10.217827 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:10.217798 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-p96xl"] Jul 03 20:38:10.218668 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:38:10.218637 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b625e8_c048_4008_8ef5_c94c73f6c5a3.slice/crio-529345c70b03f8cdf5915e0d529122f0f56123419da21d62d5dc9c52d469c425 WatchSource:0}: Error finding container 529345c70b03f8cdf5915e0d529122f0f56123419da21d62d5dc9c52d469c425: Status 404 returned error can't find the container with id 529345c70b03f8cdf5915e0d529122f0f56123419da21d62d5dc9c52d469c425 Jul 03 20:38:10.359811 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:10.359700 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6tfkc" event={"ID":"91a7f796-addb-4293-9e54-07cd9433cce3","Type":"ContainerStarted","Data":"2675be0a9501a77fc4cf194d43bdc6a598fa3e44ef71d6e15140102b0e504b8b"} Jul 03 20:38:10.359964 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:10.359825 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-6tfkc" Jul 03 20:38:10.361493 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:10.361452 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" event={"ID":"47b625e8-c048-4008-8ef5-c94c73f6c5a3","Type":"ContainerStarted","Data":"529345c70b03f8cdf5915e0d529122f0f56123419da21d62d5dc9c52d469c425"} Jul 03 20:38:10.362926 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:10.362886 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-wd9c5" event={"ID":"e0ed9b90-a468-468c-be69-e7e41de57781","Type":"ContainerStarted","Data":"8ecb25ee4b65da8b7e8c7e669c5f5f7f0036b6c829bf41ed3808e76b11168293"} Jul 03 20:38:10.364715 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:10.364658 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5546b876fd-gqk58" event={"ID":"4a2d0db1-3f77-4931-b456-58ea5a1f46de","Type":"ContainerStarted","Data":"f2cd15a08f6a156899e353fc2d0dca05e0ab3142a93adce77214470a47226ae4"} Jul 03 20:38:10.380465 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:10.380211 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-6tfkc" podStartSLOduration=2.347614198 podStartE2EDuration="5.38019282s" totalImagesPullingTime="3.032578622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:38:10.379658258 +0000 UTC m=+356.837930710" watchObservedRunningTime="2026-07-03 20:38:10.38019282 +0000 UTC m=+356.838465261" Jul 03 20:38:10.409434 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:10.409375 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5546b876fd-gqk58" podStartSLOduration=1.273653525 podStartE2EDuration="5.409352163s" totalImagesPullingTime="4.135698638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:38:10.408382461 +0000 UTC m=+356.866654900" watchObservedRunningTime="2026-07-03 20:38:10.409352163 +0000 UTC m=+356.867624603" Jul 03 20:38:12.851779 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:12.851726 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 03 20:38:12.852044 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:12.851820 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 03 20:38:13.384242 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:13.384202 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" event={"ID":"47b625e8-c048-4008-8ef5-c94c73f6c5a3","Type":"ContainerStarted","Data":"e564ecb20af9c358925ed304ad5d1cdd81fcaeed933d26765879416b015c2fad"} Jul 03 20:38:13.384431 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:13.384408 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:13.385949 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:13.385923 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-wd9c5" event={"ID":"e0ed9b90-a468-468c-be69-e7e41de57781","Type":"ContainerStarted","Data":"7789e56d528712aa139d6df1bf9609c4bfeedabd9b939a660d3c86468db872c1"} Jul 03 20:38:13.386058 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:13.385926 2582 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-p96xl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 03 20:38:13.386058 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:13.385991 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" podUID="47b625e8-c048-4008-8ef5-c94c73f6c5a3" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 03 20:38:13.386140 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:13.386056 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-wd9c5" Jul 03 20:38:13.407267 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:13.407213 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" podStartSLOduration=2.776503442 podStartE2EDuration="5.407190363s" totalImagesPullingTime="2.630686921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:38:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:38:13.405739622 +0000 UTC m=+359.864012064" watchObservedRunningTime="2026-07-03 20:38:13.407190363 +0000 UTC m=+359.865462803" Jul 03 20:38:13.430904 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:13.430852 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-wd9c5" podStartSLOduration=3.78073061 podStartE2EDuration="6.430834134s" totalImagesPullingTime="2.650103524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:38:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:38:13.427681978 +0000 UTC m=+359.885954419" watchObservedRunningTime="2026-07-03 20:38:13.430834134 +0000 UTC m=+359.889106573" Jul 03 20:38:14.390678 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:14.390652 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:17.577008 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.576974 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp"] Jul 03 20:38:17.581336 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.581317 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp" Jul 03 20:38:17.585067 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.585048 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 03 20:38:17.585186 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.585078 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-w9h9g\"" Jul 03 20:38:17.585794 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.585776 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj"] Jul 03 20:38:17.588931 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.588914 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj" Jul 03 20:38:17.592257 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.592241 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 03 20:38:17.592570 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.592553 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-bkw2p\"" Jul 03 20:38:17.596364 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.596346 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp"] Jul 03 20:38:17.612547 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.612492 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj"] Jul 03 20:38:17.713429 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.713398 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv4mr\" (UniqueName: \"kubernetes.io/projected/b1a66c17-ccf0-4802-86b0-b16795c603a4-kube-api-access-hv4mr\") pod \"kserve-controller-manager-d6f4b65b8-tvcsp\" (UID: \"b1a66c17-ccf0-4802-86b0-b16795c603a4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp" Jul 03 20:38:17.713429 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.713432 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1a66c17-ccf0-4802-86b0-b16795c603a4-cert\") pod \"kserve-controller-manager-d6f4b65b8-tvcsp\" (UID: \"b1a66c17-ccf0-4802-86b0-b16795c603a4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp" Jul 03 20:38:17.713632 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.713467 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb5c5346-91f7-4f20-b3bf-e2beb1aec43b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k27gj\" (UID: \"bb5c5346-91f7-4f20-b3bf-e2beb1aec43b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj" Jul 03 20:38:17.713632 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.713513 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d2tb\" (UniqueName: \"kubernetes.io/projected/bb5c5346-91f7-4f20-b3bf-e2beb1aec43b-kube-api-access-2d2tb\") pod \"llmisvc-controller-manager-777c6bd7d5-k27gj\" (UID: \"bb5c5346-91f7-4f20-b3bf-e2beb1aec43b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj" Jul 03 20:38:17.814851 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.814813 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv4mr\" (UniqueName: \"kubernetes.io/projected/b1a66c17-ccf0-4802-86b0-b16795c603a4-kube-api-access-hv4mr\") pod \"kserve-controller-manager-d6f4b65b8-tvcsp\" (UID: \"b1a66c17-ccf0-4802-86b0-b16795c603a4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp" Jul 03 20:38:17.814851 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.814856 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1a66c17-ccf0-4802-86b0-b16795c603a4-cert\") pod \"kserve-controller-manager-d6f4b65b8-tvcsp\" (UID: \"b1a66c17-ccf0-4802-86b0-b16795c603a4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp" Jul 03 20:38:17.815102 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.814893 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb5c5346-91f7-4f20-b3bf-e2beb1aec43b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k27gj\" (UID: \"bb5c5346-91f7-4f20-b3bf-e2beb1aec43b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj" Jul 03 20:38:17.815102 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.814942 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d2tb\" (UniqueName: \"kubernetes.io/projected/bb5c5346-91f7-4f20-b3bf-e2beb1aec43b-kube-api-access-2d2tb\") pod \"llmisvc-controller-manager-777c6bd7d5-k27gj\" (UID: \"bb5c5346-91f7-4f20-b3bf-e2beb1aec43b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj" Jul 03 20:38:17.815102 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:38:17.814988 2582 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 03 20:38:17.815102 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:38:17.815063 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb5c5346-91f7-4f20-b3bf-e2beb1aec43b-cert podName:bb5c5346-91f7-4f20-b3bf-e2beb1aec43b nodeName:}" failed. No retries permitted until 2026-07-03 20:38:18.315042567 +0000 UTC m=+364.773314985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bb5c5346-91f7-4f20-b3bf-e2beb1aec43b-cert") pod "llmisvc-controller-manager-777c6bd7d5-k27gj" (UID: "bb5c5346-91f7-4f20-b3bf-e2beb1aec43b") : secret "llmisvc-webhook-server-cert" not found Jul 03 20:38:17.817446 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.817425 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1a66c17-ccf0-4802-86b0-b16795c603a4-cert\") pod \"kserve-controller-manager-d6f4b65b8-tvcsp\" (UID: \"b1a66c17-ccf0-4802-86b0-b16795c603a4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp" Jul 03 20:38:17.829815 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.829737 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv4mr\" (UniqueName: \"kubernetes.io/projected/b1a66c17-ccf0-4802-86b0-b16795c603a4-kube-api-access-hv4mr\") pod \"kserve-controller-manager-d6f4b65b8-tvcsp\" (UID: \"b1a66c17-ccf0-4802-86b0-b16795c603a4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp" Jul 03 20:38:17.830300 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.830283 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d2tb\" (UniqueName: \"kubernetes.io/projected/bb5c5346-91f7-4f20-b3bf-e2beb1aec43b-kube-api-access-2d2tb\") pod \"llmisvc-controller-manager-777c6bd7d5-k27gj\" (UID: \"bb5c5346-91f7-4f20-b3bf-e2beb1aec43b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj" Jul 03 20:38:17.894474 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:17.894450 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp" Jul 03 20:38:18.051369 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:18.051290 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp"] Jul 03 20:38:18.053305 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:38:18.053279 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1a66c17_ccf0_4802_86b0_b16795c603a4.slice/crio-be6b8dacaf5bbd3c312ef6507bed271d5a821c394343873a77aa43fecfb74558 WatchSource:0}: Error finding container be6b8dacaf5bbd3c312ef6507bed271d5a821c394343873a77aa43fecfb74558: Status 404 returned error can't find the container with id be6b8dacaf5bbd3c312ef6507bed271d5a821c394343873a77aa43fecfb74558 Jul 03 20:38:18.319135 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:18.319093 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb5c5346-91f7-4f20-b3bf-e2beb1aec43b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k27gj\" (UID: \"bb5c5346-91f7-4f20-b3bf-e2beb1aec43b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj" Jul 03 20:38:18.321588 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:18.321564 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb5c5346-91f7-4f20-b3bf-e2beb1aec43b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k27gj\" (UID: \"bb5c5346-91f7-4f20-b3bf-e2beb1aec43b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj" Jul 03 20:38:18.406464 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:18.406432 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp" event={"ID":"b1a66c17-ccf0-4802-86b0-b16795c603a4","Type":"ContainerStarted","Data":"be6b8dacaf5bbd3c312ef6507bed271d5a821c394343873a77aa43fecfb74558"} Jul 03 20:38:18.504497 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:18.504464 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj" Jul 03 20:38:18.635208 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:18.635183 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj"] Jul 03 20:38:18.637561 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:38:18.637536 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbb5c5346_91f7_4f20_b3bf_e2beb1aec43b.slice/crio-0a2763db33155e4b68b649b4ceb670606cfcf8444bb75372bff2bef5becee942 WatchSource:0}: Error finding container 0a2763db33155e4b68b649b4ceb670606cfcf8444bb75372bff2bef5becee942: Status 404 returned error can't find the container with id 0a2763db33155e4b68b649b4ceb670606cfcf8444bb75372bff2bef5becee942 Jul 03 20:38:19.412325 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:19.412277 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj" event={"ID":"bb5c5346-91f7-4f20-b3bf-e2beb1aec43b","Type":"ContainerStarted","Data":"0a2763db33155e4b68b649b4ceb670606cfcf8444bb75372bff2bef5becee942"} Jul 03 20:38:19.788719 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:19.788609 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-p96xl"] Jul 03 20:38:19.789189 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:19.788871 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" podUID="47b625e8-c048-4008-8ef5-c94c73f6c5a3" containerName="discovery" containerID="cri-o://e564ecb20af9c358925ed304ad5d1cdd81fcaeed933d26765879416b015c2fad" gracePeriod=30 Jul 03 20:38:20.391423 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.391383 2582 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-p96xl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.20:8080/ready\": dial tcp 10.133.0.20:8080: connect: connection refused" start-of-body= Jul 03 20:38:20.391563 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.391441 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" podUID="47b625e8-c048-4008-8ef5-c94c73f6c5a3" containerName="discovery" probeResult="failure" output="Get \"http://10.133.0.20:8080/ready\": dial tcp 10.133.0.20:8080: connect: connection refused" Jul 03 20:38:20.418717 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.418687 2582 generic.go:356] "Generic (PLEG): container finished" podID="47b625e8-c048-4008-8ef5-c94c73f6c5a3" containerID="e564ecb20af9c358925ed304ad5d1cdd81fcaeed933d26765879416b015c2fad" exitCode=0 Jul 03 20:38:20.418887 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.418726 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" event={"ID":"47b625e8-c048-4008-8ef5-c94c73f6c5a3","Type":"ContainerDied","Data":"e564ecb20af9c358925ed304ad5d1cdd81fcaeed933d26765879416b015c2fad"} Jul 03 20:38:20.566834 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.566806 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:20.639253 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.639186 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-kubeconfig\") pod \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " Jul 03 20:38:20.639253 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.639233 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-cacerts\") pod \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " Jul 03 20:38:20.639420 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.639334 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/47b625e8-c048-4008-8ef5-c94c73f6c5a3-local-certs\") pod \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " Jul 03 20:38:20.639420 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.639365 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-token\") pod \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " Jul 03 20:38:20.639420 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.639415 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hz7v\" (UniqueName: \"kubernetes.io/projected/47b625e8-c048-4008-8ef5-c94c73f6c5a3-kube-api-access-8hz7v\") pod \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " Jul 03 20:38:20.639550 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.639441 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-csr-ca-configmap\") pod \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " Jul 03 20:38:20.639550 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.639479 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-csr-dns-cert\") pod \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\" (UID: \"47b625e8-c048-4008-8ef5-c94c73f6c5a3\") " Jul 03 20:38:20.640279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.640107 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "47b625e8-c048-4008-8ef5-c94c73f6c5a3" (UID: "47b625e8-c048-4008-8ef5-c94c73f6c5a3"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:38:20.641946 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.641898 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-token" (OuterVolumeSpecName: "istio-token") pod "47b625e8-c048-4008-8ef5-c94c73f6c5a3" (UID: "47b625e8-c048-4008-8ef5-c94c73f6c5a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:38:20.642060 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.641968 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "47b625e8-c048-4008-8ef5-c94c73f6c5a3" (UID: "47b625e8-c048-4008-8ef5-c94c73f6c5a3"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:38:20.642128 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.642103 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-cacerts" (OuterVolumeSpecName: "cacerts") pod "47b625e8-c048-4008-8ef5-c94c73f6c5a3" (UID: "47b625e8-c048-4008-8ef5-c94c73f6c5a3"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:38:20.642352 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.642333 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47b625e8-c048-4008-8ef5-c94c73f6c5a3-local-certs" (OuterVolumeSpecName: "local-certs") pod "47b625e8-c048-4008-8ef5-c94c73f6c5a3" (UID: "47b625e8-c048-4008-8ef5-c94c73f6c5a3"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:38:20.642420 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.642348 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "47b625e8-c048-4008-8ef5-c94c73f6c5a3" (UID: "47b625e8-c048-4008-8ef5-c94c73f6c5a3"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:38:20.642456 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.642417 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47b625e8-c048-4008-8ef5-c94c73f6c5a3-kube-api-access-8hz7v" (OuterVolumeSpecName: "kube-api-access-8hz7v") pod "47b625e8-c048-4008-8ef5-c94c73f6c5a3" (UID: "47b625e8-c048-4008-8ef5-c94c73f6c5a3"). InnerVolumeSpecName "kube-api-access-8hz7v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:38:20.741316 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.741217 2582 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/47b625e8-c048-4008-8ef5-c94c73f6c5a3-local-certs\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:38:20.741316 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.741248 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-token\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:38:20.741316 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.741262 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hz7v\" (UniqueName: \"kubernetes.io/projected/47b625e8-c048-4008-8ef5-c94c73f6c5a3-kube-api-access-8hz7v\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:38:20.741593 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.741276 2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-csr-ca-configmap\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:38:20.741593 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.741350 2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-csr-dns-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:38:20.741593 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.741363 2582 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-istio-kubeconfig\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:38:20.741593 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:20.741375 2582 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/47b625e8-c048-4008-8ef5-c94c73f6c5a3-cacerts\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:38:21.376790 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:21.376739 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-6tfkc" Jul 03 20:38:21.427155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:21.427122 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" event={"ID":"47b625e8-c048-4008-8ef5-c94c73f6c5a3","Type":"ContainerDied","Data":"529345c70b03f8cdf5915e0d529122f0f56123419da21d62d5dc9c52d469c425"} Jul 03 20:38:21.427305 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:21.427187 2582 scope.go:117] "RemoveContainer" containerID="e564ecb20af9c358925ed304ad5d1cdd81fcaeed933d26765879416b015c2fad" Jul 03 20:38:21.427350 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:21.427339 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-p96xl" Jul 03 20:38:21.431600 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:21.431539 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp" event={"ID":"b1a66c17-ccf0-4802-86b0-b16795c603a4","Type":"ContainerStarted","Data":"734801fdd2a279eb41d12df10663ef5a0a93fa9c65abc0d54b2e635412fa22bd"} Jul 03 20:38:21.431865 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:21.431823 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp" Jul 03 20:38:21.492057 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:21.492003 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-p96xl"] Jul 03 20:38:21.493388 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:21.493363 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-p96xl"] Jul 03 20:38:21.602033 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:21.601982 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp" podStartSLOduration=2.109301452 podStartE2EDuration="4.601960934s" totalImagesPullingTime="2.492659482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:38:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:38:21.599081113 +0000 UTC m=+368.057353553" watchObservedRunningTime="2026-07-03 20:38:21.601960934 +0000 UTC m=+368.060233375" Jul 03 20:38:21.985780 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:21.985685 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47b625e8-c048-4008-8ef5-c94c73f6c5a3" path="/var/lib/kubelet/pods/47b625e8-c048-4008-8ef5-c94c73f6c5a3/volumes" Jul 03 20:38:22.437140 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:22.437106 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj" event={"ID":"bb5c5346-91f7-4f20-b3bf-e2beb1aec43b","Type":"ContainerStarted","Data":"5cfc1fa63370856aeb47f6909024265288cc8b373bcf8ff6cbc6dffbc5a08b72"} Jul 03 20:38:22.437501 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:22.437236 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj" Jul 03 20:38:22.456179 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:22.456138 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj" podStartSLOduration=2.5149894550000003 podStartE2EDuration="5.456122305s" totalImagesPullingTime="2.94113285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:38:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:38:22.455291905 +0000 UTC m=+368.913564344" watchObservedRunningTime="2026-07-03 20:38:22.456122305 +0000 UTC m=+368.914394744" Jul 03 20:38:24.394257 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:24.394224 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-wd9c5" Jul 03 20:38:52.442418 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:52.442387 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tvcsp" Jul 03 20:38:53.443455 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:38:53.443428 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k27gj" Jul 03 20:39:06.831414 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:06.831384 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gs8p6"] Jul 03 20:39:06.831796 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:06.831663 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47b625e8-c048-4008-8ef5-c94c73f6c5a3" containerName="discovery" Jul 03 20:39:06.831796 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:06.831673 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="47b625e8-c048-4008-8ef5-c94c73f6c5a3" containerName="discovery" Jul 03 20:39:06.831796 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:06.831716 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="47b625e8-c048-4008-8ef5-c94c73f6c5a3" containerName="discovery" Jul 03 20:39:06.833503 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:06.833488 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gs8p6" Jul 03 20:39:06.837646 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:06.837626 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 03 20:39:06.838767 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:06.838723 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-xxr4c\"" Jul 03 20:39:06.838767 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:06.838725 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 03 20:39:06.860842 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:06.860819 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gs8p6"] Jul 03 20:39:06.871676 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:06.871648 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6gds\" (UniqueName: \"kubernetes.io/projected/ec20a765-825a-4c06-a0ac-0887120f374b-kube-api-access-c6gds\") pod \"kuadrant-operator-catalog-gs8p6\" (UID: \"ec20a765-825a-4c06-a0ac-0887120f374b\") " pod="kuadrant-system/kuadrant-operator-catalog-gs8p6" Jul 03 20:39:06.972170 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:06.972138 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6gds\" (UniqueName: \"kubernetes.io/projected/ec20a765-825a-4c06-a0ac-0887120f374b-kube-api-access-c6gds\") pod \"kuadrant-operator-catalog-gs8p6\" (UID: \"ec20a765-825a-4c06-a0ac-0887120f374b\") " pod="kuadrant-system/kuadrant-operator-catalog-gs8p6" Jul 03 20:39:06.982238 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:06.982213 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6gds\" (UniqueName: \"kubernetes.io/projected/ec20a765-825a-4c06-a0ac-0887120f374b-kube-api-access-c6gds\") pod \"kuadrant-operator-catalog-gs8p6\" (UID: \"ec20a765-825a-4c06-a0ac-0887120f374b\") " pod="kuadrant-system/kuadrant-operator-catalog-gs8p6" Jul 03 20:39:07.043690 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:07.043657 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gs8p6"] Jul 03 20:39:07.043949 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:07.043935 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gs8p6" Jul 03 20:39:07.173995 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:07.173842 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gs8p6"] Jul 03 20:39:07.176551 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:39:07.176522 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec20a765_825a_4c06_a0ac_0887120f374b.slice/crio-807d04831ad5cdf21bf0651723b3e1721914a9b5087df08ded38e8363343ab74 WatchSource:0}: Error finding container 807d04831ad5cdf21bf0651723b3e1721914a9b5087df08ded38e8363343ab74: Status 404 returned error can't find the container with id 807d04831ad5cdf21bf0651723b3e1721914a9b5087df08ded38e8363343ab74 Jul 03 20:39:07.255669 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:07.255640 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8m99h"] Jul 03 20:39:07.258304 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:07.258289 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8m99h" Jul 03 20:39:07.267722 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:07.267687 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8m99h"] Jul 03 20:39:07.274502 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:07.274481 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5qdr\" (UniqueName: \"kubernetes.io/projected/0f89b024-bb82-4a03-85e3-df0585ce2682-kube-api-access-j5qdr\") pod \"kuadrant-operator-catalog-8m99h\" (UID: \"0f89b024-bb82-4a03-85e3-df0585ce2682\") " pod="kuadrant-system/kuadrant-operator-catalog-8m99h" Jul 03 20:39:07.375447 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:07.375363 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5qdr\" (UniqueName: \"kubernetes.io/projected/0f89b024-bb82-4a03-85e3-df0585ce2682-kube-api-access-j5qdr\") pod \"kuadrant-operator-catalog-8m99h\" (UID: \"0f89b024-bb82-4a03-85e3-df0585ce2682\") " pod="kuadrant-system/kuadrant-operator-catalog-8m99h" Jul 03 20:39:07.384227 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:07.384197 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5qdr\" (UniqueName: \"kubernetes.io/projected/0f89b024-bb82-4a03-85e3-df0585ce2682-kube-api-access-j5qdr\") pod \"kuadrant-operator-catalog-8m99h\" (UID: \"0f89b024-bb82-4a03-85e3-df0585ce2682\") " pod="kuadrant-system/kuadrant-operator-catalog-8m99h" Jul 03 20:39:07.567796 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:07.567763 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8m99h" Jul 03 20:39:07.624554 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:07.624464 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gs8p6" event={"ID":"ec20a765-825a-4c06-a0ac-0887120f374b","Type":"ContainerStarted","Data":"807d04831ad5cdf21bf0651723b3e1721914a9b5087df08ded38e8363343ab74"} Jul 03 20:39:07.707797 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:07.707739 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8m99h"] Jul 03 20:39:07.783587 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:39:07.783551 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f89b024_bb82_4a03_85e3_df0585ce2682.slice/crio-52d9dceba003a73f5e1c5c4ddd2ab807c1e7ba71bbfcb916532f5a251273ad39 WatchSource:0}: Error finding container 52d9dceba003a73f5e1c5c4ddd2ab807c1e7ba71bbfcb916532f5a251273ad39: Status 404 returned error can't find the container with id 52d9dceba003a73f5e1c5c4ddd2ab807c1e7ba71bbfcb916532f5a251273ad39 Jul 03 20:39:08.630336 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:08.630298 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8m99h" event={"ID":"0f89b024-bb82-4a03-85e3-df0585ce2682","Type":"ContainerStarted","Data":"52d9dceba003a73f5e1c5c4ddd2ab807c1e7ba71bbfcb916532f5a251273ad39"} Jul 03 20:39:09.635070 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:09.635037 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8m99h" event={"ID":"0f89b024-bb82-4a03-85e3-df0585ce2682","Type":"ContainerStarted","Data":"596eaf50bc9196c6443e507d32b399435f2d2851c111382d5553f45da49a5f60"} Jul 03 20:39:09.636535 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:09.636510 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gs8p6" event={"ID":"ec20a765-825a-4c06-a0ac-0887120f374b","Type":"ContainerStarted","Data":"5177acbfafd9153af9de74a53f8b0971a446d8437410c44078b1912988bda442"} Jul 03 20:39:09.636649 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:09.636571 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-gs8p6" podUID="ec20a765-825a-4c06-a0ac-0887120f374b" containerName="registry-server" containerID="cri-o://5177acbfafd9153af9de74a53f8b0971a446d8437410c44078b1912988bda442" gracePeriod=2 Jul 03 20:39:09.652976 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:09.652937 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-8m99h" podStartSLOduration=1.545949097 podStartE2EDuration="2.652923738s" totalImagesPullingTime="1.106974641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:39:09.651392092 +0000 UTC m=+416.109664531" watchObservedRunningTime="2026-07-03 20:39:09.652923738 +0000 UTC m=+416.111196177" Jul 03 20:39:09.669035 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:09.668998 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-gs8p6" podStartSLOduration=1.955446531 podStartE2EDuration="3.668985974s" totalImagesPullingTime="1.713539443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:39:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:39:09.668222123 +0000 UTC m=+416.126494562" watchObservedRunningTime="2026-07-03 20:39:09.668985974 +0000 UTC m=+416.127258413" Jul 03 20:39:09.871815 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:09.871791 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gs8p6" Jul 03 20:39:09.894021 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:09.893963 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6gds\" (UniqueName: \"kubernetes.io/projected/ec20a765-825a-4c06-a0ac-0887120f374b-kube-api-access-c6gds\") pod \"ec20a765-825a-4c06-a0ac-0887120f374b\" (UID: \"ec20a765-825a-4c06-a0ac-0887120f374b\") " Jul 03 20:39:09.896120 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:09.896097 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec20a765-825a-4c06-a0ac-0887120f374b-kube-api-access-c6gds" (OuterVolumeSpecName: "kube-api-access-c6gds") pod "ec20a765-825a-4c06-a0ac-0887120f374b" (UID: "ec20a765-825a-4c06-a0ac-0887120f374b"). InnerVolumeSpecName "kube-api-access-c6gds". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:39:09.995449 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:09.995420 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6gds\" (UniqueName: \"kubernetes.io/projected/ec20a765-825a-4c06-a0ac-0887120f374b-kube-api-access-c6gds\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:39:10.641923 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:10.641889 2582 generic.go:356] "Generic (PLEG): container finished" podID="ec20a765-825a-4c06-a0ac-0887120f374b" containerID="5177acbfafd9153af9de74a53f8b0971a446d8437410c44078b1912988bda442" exitCode=0 Jul 03 20:39:10.642390 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:10.641931 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gs8p6" Jul 03 20:39:10.642390 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:10.641974 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gs8p6" event={"ID":"ec20a765-825a-4c06-a0ac-0887120f374b","Type":"ContainerDied","Data":"5177acbfafd9153af9de74a53f8b0971a446d8437410c44078b1912988bda442"} Jul 03 20:39:10.642390 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:10.642028 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gs8p6" event={"ID":"ec20a765-825a-4c06-a0ac-0887120f374b","Type":"ContainerDied","Data":"807d04831ad5cdf21bf0651723b3e1721914a9b5087df08ded38e8363343ab74"} Jul 03 20:39:10.642390 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:10.642059 2582 scope.go:117] "RemoveContainer" containerID="5177acbfafd9153af9de74a53f8b0971a446d8437410c44078b1912988bda442" Jul 03 20:39:10.650170 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:10.650153 2582 scope.go:117] "RemoveContainer" containerID="5177acbfafd9153af9de74a53f8b0971a446d8437410c44078b1912988bda442" Jul 03 20:39:10.650415 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:39:10.650399 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5177acbfafd9153af9de74a53f8b0971a446d8437410c44078b1912988bda442\": container with ID starting with 5177acbfafd9153af9de74a53f8b0971a446d8437410c44078b1912988bda442 not found: ID does not exist" containerID="5177acbfafd9153af9de74a53f8b0971a446d8437410c44078b1912988bda442" Jul 03 20:39:10.650493 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:10.650425 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5177acbfafd9153af9de74a53f8b0971a446d8437410c44078b1912988bda442"} err="failed to get container status \"5177acbfafd9153af9de74a53f8b0971a446d8437410c44078b1912988bda442\": rpc error: code = NotFound desc = could not find container \"5177acbfafd9153af9de74a53f8b0971a446d8437410c44078b1912988bda442\": container with ID starting with 5177acbfafd9153af9de74a53f8b0971a446d8437410c44078b1912988bda442 not found: ID does not exist" Jul 03 20:39:10.656936 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:10.656916 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gs8p6"] Jul 03 20:39:10.659087 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:10.659069 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gs8p6"] Jul 03 20:39:11.985325 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:11.985293 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec20a765-825a-4c06-a0ac-0887120f374b" path="/var/lib/kubelet/pods/ec20a765-825a-4c06-a0ac-0887120f374b/volumes" Jul 03 20:39:17.568878 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:17.568845 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-8m99h" Jul 03 20:39:17.569299 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:17.568957 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-8m99h" Jul 03 20:39:17.591176 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:17.591154 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-8m99h" Jul 03 20:39:17.692658 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:17.692634 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-8m99h" Jul 03 20:39:38.166008 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:38.165930 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-d8wbw"] Jul 03 20:39:38.166460 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:38.166228 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec20a765-825a-4c06-a0ac-0887120f374b" containerName="registry-server" Jul 03 20:39:38.166460 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:38.166243 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec20a765-825a-4c06-a0ac-0887120f374b" containerName="registry-server" Jul 03 20:39:38.166460 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:38.166306 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec20a765-825a-4c06-a0ac-0887120f374b" containerName="registry-server" Jul 03 20:39:38.169486 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:38.169464 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d8wbw" Jul 03 20:39:38.171999 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:38.171975 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 03 20:39:38.172391 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:38.172374 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-8sn5k\"" Jul 03 20:39:38.180408 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:38.180378 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-d8wbw"] Jul 03 20:39:38.201260 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:38.201223 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86hw9\" (UniqueName: \"kubernetes.io/projected/4abbcd18-88af-4e83-9fef-73920817cb2e-kube-api-access-86hw9\") pod \"dns-operator-controller-manager-864dd78c4c-d8wbw\" (UID: \"4abbcd18-88af-4e83-9fef-73920817cb2e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d8wbw" Jul 03 20:39:38.302651 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:38.302609 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86hw9\" (UniqueName: \"kubernetes.io/projected/4abbcd18-88af-4e83-9fef-73920817cb2e-kube-api-access-86hw9\") pod \"dns-operator-controller-manager-864dd78c4c-d8wbw\" (UID: \"4abbcd18-88af-4e83-9fef-73920817cb2e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d8wbw" Jul 03 20:39:38.316416 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:38.316392 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86hw9\" (UniqueName: \"kubernetes.io/projected/4abbcd18-88af-4e83-9fef-73920817cb2e-kube-api-access-86hw9\") pod \"dns-operator-controller-manager-864dd78c4c-d8wbw\" (UID: \"4abbcd18-88af-4e83-9fef-73920817cb2e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d8wbw" Jul 03 20:39:38.479616 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:38.479527 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d8wbw" Jul 03 20:39:38.612501 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:38.612479 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-d8wbw"] Jul 03 20:39:38.614860 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:39:38.614829 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4abbcd18_88af_4e83_9fef_73920817cb2e.slice/crio-934d0c0df299b21dc26cd328d198456ae9c2e2f0b5fee12f448ce59284cb87e1 WatchSource:0}: Error finding container 934d0c0df299b21dc26cd328d198456ae9c2e2f0b5fee12f448ce59284cb87e1: Status 404 returned error can't find the container with id 934d0c0df299b21dc26cd328d198456ae9c2e2f0b5fee12f448ce59284cb87e1 Jul 03 20:39:38.752869 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:38.752789 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d8wbw" event={"ID":"4abbcd18-88af-4e83-9fef-73920817cb2e","Type":"ContainerStarted","Data":"934d0c0df299b21dc26cd328d198456ae9c2e2f0b5fee12f448ce59284cb87e1"} Jul 03 20:39:40.763468 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:40.763429 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d8wbw" event={"ID":"4abbcd18-88af-4e83-9fef-73920817cb2e","Type":"ContainerStarted","Data":"2570630b38cddfcf0efe2eb0edfa20043f544083be46d637d42a91d92b3aabc3"} Jul 03 20:39:40.763882 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:40.763560 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d8wbw" Jul 03 20:39:40.790236 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:40.790187 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d8wbw" podStartSLOduration=1.056948497 podStartE2EDuration="2.790172434s" totalImagesPullingTime="1.733223937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:39:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:39:40.788471593 +0000 UTC m=+447.246744032" watchObservedRunningTime="2026-07-03 20:39:40.790172434 +0000 UTC m=+447.248444908" Jul 03 20:39:42.236958 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:42.236920 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-r8zvb"] Jul 03 20:39:42.240186 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:42.240167 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r8zvb" Jul 03 20:39:42.242918 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:42.242901 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-c9pcw\"" Jul 03 20:39:42.252783 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:42.252741 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-r8zvb"] Jul 03 20:39:42.336682 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:42.336649 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdkc4\" (UniqueName: \"kubernetes.io/projected/cd8f7050-111c-4d0b-a046-9cddcc7da06a-kube-api-access-qdkc4\") pod \"authorino-operator-75cc8d5cb7-r8zvb\" (UID: \"cd8f7050-111c-4d0b-a046-9cddcc7da06a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-r8zvb" Jul 03 20:39:42.437127 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:42.437088 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdkc4\" (UniqueName: \"kubernetes.io/projected/cd8f7050-111c-4d0b-a046-9cddcc7da06a-kube-api-access-qdkc4\") pod \"authorino-operator-75cc8d5cb7-r8zvb\" (UID: \"cd8f7050-111c-4d0b-a046-9cddcc7da06a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-r8zvb" Jul 03 20:39:42.446211 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:42.446186 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdkc4\" (UniqueName: \"kubernetes.io/projected/cd8f7050-111c-4d0b-a046-9cddcc7da06a-kube-api-access-qdkc4\") pod \"authorino-operator-75cc8d5cb7-r8zvb\" (UID: \"cd8f7050-111c-4d0b-a046-9cddcc7da06a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-r8zvb" Jul 03 20:39:42.549817 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:42.549712 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r8zvb" Jul 03 20:39:42.679096 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:42.679070 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-r8zvb"] Jul 03 20:39:42.681136 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:39:42.681109 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd8f7050_111c_4d0b_a046_9cddcc7da06a.slice/crio-51a65a9613472b77a51549c4a48979bf76cb0cbbb6cc2c510dcf7e6add00606d WatchSource:0}: Error finding container 51a65a9613472b77a51549c4a48979bf76cb0cbbb6cc2c510dcf7e6add00606d: Status 404 returned error can't find the container with id 51a65a9613472b77a51549c4a48979bf76cb0cbbb6cc2c510dcf7e6add00606d Jul 03 20:39:42.772851 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:42.772815 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r8zvb" event={"ID":"cd8f7050-111c-4d0b-a046-9cddcc7da06a","Type":"ContainerStarted","Data":"51a65a9613472b77a51549c4a48979bf76cb0cbbb6cc2c510dcf7e6add00606d"} Jul 03 20:39:45.580336 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:45.580304 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k"] Jul 03 20:39:45.583537 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:45.583518 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" Jul 03 20:39:45.586818 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:45.586795 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-j9ndc\"" Jul 03 20:39:45.599820 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:45.599797 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k"] Jul 03 20:39:45.662561 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:45.662532 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlwd4\" (UniqueName: \"kubernetes.io/projected/56fec6b1-f650-4dbe-b6c8-a6263870181c-kube-api-access-zlwd4\") pod \"limitador-operator-controller-manager-859d7b78bd-4526k\" (UID: \"56fec6b1-f650-4dbe-b6c8-a6263870181c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" Jul 03 20:39:45.763722 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:45.763683 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlwd4\" (UniqueName: \"kubernetes.io/projected/56fec6b1-f650-4dbe-b6c8-a6263870181c-kube-api-access-zlwd4\") pod \"limitador-operator-controller-manager-859d7b78bd-4526k\" (UID: \"56fec6b1-f650-4dbe-b6c8-a6263870181c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" Jul 03 20:39:45.776067 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:45.776034 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlwd4\" (UniqueName: \"kubernetes.io/projected/56fec6b1-f650-4dbe-b6c8-a6263870181c-kube-api-access-zlwd4\") pod \"limitador-operator-controller-manager-859d7b78bd-4526k\" (UID: \"56fec6b1-f650-4dbe-b6c8-a6263870181c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" Jul 03 20:39:45.788232 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:45.788202 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r8zvb" event={"ID":"cd8f7050-111c-4d0b-a046-9cddcc7da06a","Type":"ContainerStarted","Data":"d520ff4fa0bced314ea4e59cc7dc5b8bb5d653f27d5a573b8b15625fbc59c53f"} Jul 03 20:39:45.788365 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:45.788287 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r8zvb" Jul 03 20:39:45.807421 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:45.807376 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r8zvb" podStartSLOduration=1.705036239 podStartE2EDuration="3.807362595s" totalImagesPullingTime="2.102326356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:39:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:39:45.806267578 +0000 UTC m=+452.264540020" watchObservedRunningTime="2026-07-03 20:39:45.807362595 +0000 UTC m=+452.265635035" Jul 03 20:39:45.894012 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:45.893939 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" Jul 03 20:39:46.032061 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:46.031933 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k"] Jul 03 20:39:46.034736 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:39:46.034705 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56fec6b1_f650_4dbe_b6c8_a6263870181c.slice/crio-dbd49671b963fe560d7478cb731d7070d8e3bf8e58e6197b2f44f1445f189243 WatchSource:0}: Error finding container dbd49671b963fe560d7478cb731d7070d8e3bf8e58e6197b2f44f1445f189243: Status 404 returned error can't find the container with id dbd49671b963fe560d7478cb731d7070d8e3bf8e58e6197b2f44f1445f189243 Jul 03 20:39:46.795998 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:46.795958 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" event={"ID":"56fec6b1-f650-4dbe-b6c8-a6263870181c","Type":"ContainerStarted","Data":"dbd49671b963fe560d7478cb731d7070d8e3bf8e58e6197b2f44f1445f189243"} Jul 03 20:39:47.804923 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:47.804883 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" event={"ID":"56fec6b1-f650-4dbe-b6c8-a6263870181c","Type":"ContainerStarted","Data":"f180df957689e22685ae7bc219422aea5245666df7149ca3df2eca32122b085a"} Jul 03 20:39:47.805323 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:47.805017 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" Jul 03 20:39:47.824451 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:47.824401 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" podStartSLOduration=1.26925806 podStartE2EDuration="2.824384201s" totalImagesPullingTime="1.555126141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:39:47.82296826 +0000 UTC m=+454.281240699" watchObservedRunningTime="2026-07-03 20:39:47.824384201 +0000 UTC m=+454.282656641" Jul 03 20:39:50.193306 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:50.193275 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg"] Jul 03 20:39:50.196509 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:50.196494 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:39:50.199510 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:50.199489 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-pwffs\"" Jul 03 20:39:50.221109 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:50.221079 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg"] Jul 03 20:39:50.300586 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:50.300547 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6678d1ce-f2bf-41fb-bdc4-047fd65c6030-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w6kvg\" (UID: \"6678d1ce-f2bf-41fb-bdc4-047fd65c6030\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:39:50.300737 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:50.300633 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5cgc\" (UniqueName: \"kubernetes.io/projected/6678d1ce-f2bf-41fb-bdc4-047fd65c6030-kube-api-access-k5cgc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w6kvg\" (UID: \"6678d1ce-f2bf-41fb-bdc4-047fd65c6030\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:39:50.401350 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:50.401321 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5cgc\" (UniqueName: \"kubernetes.io/projected/6678d1ce-f2bf-41fb-bdc4-047fd65c6030-kube-api-access-k5cgc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w6kvg\" (UID: \"6678d1ce-f2bf-41fb-bdc4-047fd65c6030\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:39:50.401476 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:50.401368 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6678d1ce-f2bf-41fb-bdc4-047fd65c6030-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w6kvg\" (UID: \"6678d1ce-f2bf-41fb-bdc4-047fd65c6030\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:39:50.401729 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:50.401711 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6678d1ce-f2bf-41fb-bdc4-047fd65c6030-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w6kvg\" (UID: \"6678d1ce-f2bf-41fb-bdc4-047fd65c6030\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:39:50.409872 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:50.409850 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5cgc\" (UniqueName: \"kubernetes.io/projected/6678d1ce-f2bf-41fb-bdc4-047fd65c6030-kube-api-access-k5cgc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w6kvg\" (UID: \"6678d1ce-f2bf-41fb-bdc4-047fd65c6030\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:39:50.506718 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:50.506645 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:39:50.637628 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:50.637599 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg"] Jul 03 20:39:50.640713 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:39:50.640689 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6678d1ce_f2bf_41fb_bdc4_047fd65c6030.slice/crio-394351917ffbdf46124c2a7382cebaaf2a7352dd0f72f6d42f9bdc2d2736d74c WatchSource:0}: Error finding container 394351917ffbdf46124c2a7382cebaaf2a7352dd0f72f6d42f9bdc2d2736d74c: Status 404 returned error can't find the container with id 394351917ffbdf46124c2a7382cebaaf2a7352dd0f72f6d42f9bdc2d2736d74c Jul 03 20:39:50.818661 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:50.818572 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" event={"ID":"6678d1ce-f2bf-41fb-bdc4-047fd65c6030","Type":"ContainerStarted","Data":"394351917ffbdf46124c2a7382cebaaf2a7352dd0f72f6d42f9bdc2d2736d74c"} Jul 03 20:39:51.770873 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:51.770836 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d8wbw" Jul 03 20:39:56.798573 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:56.798542 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-r8zvb" Jul 03 20:39:56.847886 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:56.847854 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" event={"ID":"6678d1ce-f2bf-41fb-bdc4-047fd65c6030","Type":"ContainerStarted","Data":"d6d0e6dfc667497404dae591f7aeecc6d065cc04dc54f2c59ae848975acb8a3a"} Jul 03 20:39:56.848030 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:56.847982 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:39:56.865786 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:56.865705 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" podStartSLOduration=1.6115716450000002 podStartE2EDuration="6.865683477s" totalImagesPullingTime="5.254111832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:39:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:39:56.864854789 +0000 UTC m=+463.323127225" watchObservedRunningTime="2026-07-03 20:39:56.865683477 +0000 UTC m=+463.323955918" Jul 03 20:39:58.811251 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:39:58.811217 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" Jul 03 20:40:07.853491 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:07.853463 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:40:09.589434 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.589406 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh"] Jul 03 20:40:09.592585 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.592570 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" Jul 03 20:40:09.596650 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.596624 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg"] Jul 03 20:40:09.596805 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.596782 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" podUID="6678d1ce-f2bf-41fb-bdc4-047fd65c6030" containerName="manager" containerID="cri-o://d6d0e6dfc667497404dae591f7aeecc6d065cc04dc54f2c59ae848975acb8a3a" gracePeriod=2 Jul 03 20:40:09.606882 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.606819 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh"] Jul 03 20:40:09.610422 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.610399 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg"] Jul 03 20:40:09.625316 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.625291 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k"] Jul 03 20:40:09.625494 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.625475 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" podUID="56fec6b1-f650-4dbe-b6c8-a6263870181c" containerName="manager" containerID="cri-o://f180df957689e22685ae7bc219422aea5245666df7149ca3df2eca32122b085a" gracePeriod=2 Jul 03 20:40:09.629080 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.629056 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k"] Jul 03 20:40:09.645198 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:40:09.645159 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w6kvg\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="6678d1ce-f2bf-41fb-bdc4-047fd65c6030" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:40:09.647982 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:40:09.647945 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4526k\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="56fec6b1-f650-4dbe-b6c8-a6263870181c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" Jul 03 20:40:09.657528 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.657501 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-slzfp"] Jul 03 20:40:09.657924 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.657906 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56fec6b1-f650-4dbe-b6c8-a6263870181c" containerName="manager" Jul 03 20:40:09.658011 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.657926 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="56fec6b1-f650-4dbe-b6c8-a6263870181c" containerName="manager" Jul 03 20:40:09.658011 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.657946 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6678d1ce-f2bf-41fb-bdc4-047fd65c6030" containerName="manager" Jul 03 20:40:09.658011 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.657954 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6678d1ce-f2bf-41fb-bdc4-047fd65c6030" containerName="manager" Jul 03 20:40:09.658155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.658055 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6678d1ce-f2bf-41fb-bdc4-047fd65c6030" containerName="manager" Jul 03 20:40:09.658155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.658070 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="56fec6b1-f650-4dbe-b6c8-a6263870181c" containerName="manager" Jul 03 20:40:09.660938 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.660920 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-slzfp" Jul 03 20:40:09.683619 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.683592 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-slzfp"] Jul 03 20:40:09.696250 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:40:09.696220 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w6kvg\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="6678d1ce-f2bf-41fb-bdc4-047fd65c6030" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:40:09.699326 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:40:09.699299 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4526k\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="56fec6b1-f650-4dbe-b6c8-a6263870181c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" Jul 03 20:40:09.760660 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.760634 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zhk9\" (UniqueName: \"kubernetes.io/projected/aa437708-a4ed-4bf5-a49d-8deab166366b-kube-api-access-5zhk9\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mnlmh\" (UID: \"aa437708-a4ed-4bf5-a49d-8deab166366b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" Jul 03 20:40:09.760794 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.760674 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp76r\" (UniqueName: \"kubernetes.io/projected/54a81c57-f8f4-4dd2-af7a-bc3e3ea143ec-kube-api-access-zp76r\") pod \"limitador-operator-controller-manager-859d7b78bd-slzfp\" (UID: \"54a81c57-f8f4-4dd2-af7a-bc3e3ea143ec\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-slzfp" Jul 03 20:40:09.760794 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.760717 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa437708-a4ed-4bf5-a49d-8deab166366b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mnlmh\" (UID: \"aa437708-a4ed-4bf5-a49d-8deab166366b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" Jul 03 20:40:09.875139 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.872360 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa437708-a4ed-4bf5-a49d-8deab166366b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mnlmh\" (UID: \"aa437708-a4ed-4bf5-a49d-8deab166366b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" Jul 03 20:40:09.875139 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.872447 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zhk9\" (UniqueName: \"kubernetes.io/projected/aa437708-a4ed-4bf5-a49d-8deab166366b-kube-api-access-5zhk9\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mnlmh\" (UID: \"aa437708-a4ed-4bf5-a49d-8deab166366b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" Jul 03 20:40:09.875139 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.872489 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp76r\" (UniqueName: \"kubernetes.io/projected/54a81c57-f8f4-4dd2-af7a-bc3e3ea143ec-kube-api-access-zp76r\") pod \"limitador-operator-controller-manager-859d7b78bd-slzfp\" (UID: \"54a81c57-f8f4-4dd2-af7a-bc3e3ea143ec\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-slzfp" Jul 03 20:40:09.875139 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.873112 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa437708-a4ed-4bf5-a49d-8deab166366b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mnlmh\" (UID: \"aa437708-a4ed-4bf5-a49d-8deab166366b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" Jul 03 20:40:09.890361 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.890337 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:40:09.892831 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.892816 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" Jul 03 20:40:09.894639 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:40:09.894614 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w6kvg\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="6678d1ce-f2bf-41fb-bdc4-047fd65c6030" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:40:09.898845 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:40:09.898823 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4526k\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="56fec6b1-f650-4dbe-b6c8-a6263870181c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" Jul 03 20:40:09.908805 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.908777 2582 generic.go:356] "Generic (PLEG): container finished" podID="6678d1ce-f2bf-41fb-bdc4-047fd65c6030" containerID="d6d0e6dfc667497404dae591f7aeecc6d065cc04dc54f2c59ae848975acb8a3a" exitCode=0 Jul 03 20:40:09.908921 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.908833 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:40:09.908921 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.908886 2582 scope.go:117] "RemoveContainer" containerID="d6d0e6dfc667497404dae591f7aeecc6d065cc04dc54f2c59ae848975acb8a3a" Jul 03 20:40:09.909082 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:40:09.909048 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w6kvg\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="6678d1ce-f2bf-41fb-bdc4-047fd65c6030" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:40:09.910567 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.910548 2582 generic.go:356] "Generic (PLEG): container finished" podID="56fec6b1-f650-4dbe-b6c8-a6263870181c" containerID="f180df957689e22685ae7bc219422aea5245666df7149ca3df2eca32122b085a" exitCode=0 Jul 03 20:40:09.910673 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.910607 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" Jul 03 20:40:09.911169 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:40:09.911145 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4526k\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="56fec6b1-f650-4dbe-b6c8-a6263870181c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" Jul 03 20:40:09.911837 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.911820 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp76r\" (UniqueName: \"kubernetes.io/projected/54a81c57-f8f4-4dd2-af7a-bc3e3ea143ec-kube-api-access-zp76r\") pod \"limitador-operator-controller-manager-859d7b78bd-slzfp\" (UID: \"54a81c57-f8f4-4dd2-af7a-bc3e3ea143ec\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-slzfp" Jul 03 20:40:09.913035 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.913018 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zhk9\" (UniqueName: \"kubernetes.io/projected/aa437708-a4ed-4bf5-a49d-8deab166366b-kube-api-access-5zhk9\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-mnlmh\" (UID: \"aa437708-a4ed-4bf5-a49d-8deab166366b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" Jul 03 20:40:09.915049 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:40:09.915027 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w6kvg\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="6678d1ce-f2bf-41fb-bdc4-047fd65c6030" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w6kvg" Jul 03 20:40:09.917321 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.917305 2582 scope.go:117] "RemoveContainer" containerID="d6d0e6dfc667497404dae591f7aeecc6d065cc04dc54f2c59ae848975acb8a3a" Jul 03 20:40:09.917398 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:40:09.917374 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4526k\" is forbidden: User \"system:node:ip-10-0-139-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-68.ec2.internal' and this object" podUID="56fec6b1-f650-4dbe-b6c8-a6263870181c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4526k" Jul 03 20:40:09.917556 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:40:09.917534 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6d0e6dfc667497404dae591f7aeecc6d065cc04dc54f2c59ae848975acb8a3a\": container with ID starting with d6d0e6dfc667497404dae591f7aeecc6d065cc04dc54f2c59ae848975acb8a3a not found: ID does not exist" containerID="d6d0e6dfc667497404dae591f7aeecc6d065cc04dc54f2c59ae848975acb8a3a" Jul 03 20:40:09.917602 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.917566 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6d0e6dfc667497404dae591f7aeecc6d065cc04dc54f2c59ae848975acb8a3a"} err="failed to get container status \"d6d0e6dfc667497404dae591f7aeecc6d065cc04dc54f2c59ae848975acb8a3a\": rpc error: code = NotFound desc = could not find container \"d6d0e6dfc667497404dae591f7aeecc6d065cc04dc54f2c59ae848975acb8a3a\": container with ID starting with d6d0e6dfc667497404dae591f7aeecc6d065cc04dc54f2c59ae848975acb8a3a not found: ID does not exist" Jul 03 20:40:09.917602 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.917584 2582 scope.go:117] "RemoveContainer" containerID="f180df957689e22685ae7bc219422aea5245666df7149ca3df2eca32122b085a" Jul 03 20:40:09.925109 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.925092 2582 scope.go:117] "RemoveContainer" containerID="f180df957689e22685ae7bc219422aea5245666df7149ca3df2eca32122b085a" Jul 03 20:40:09.925354 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:40:09.925336 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f180df957689e22685ae7bc219422aea5245666df7149ca3df2eca32122b085a\": container with ID starting with f180df957689e22685ae7bc219422aea5245666df7149ca3df2eca32122b085a not found: ID does not exist" containerID="f180df957689e22685ae7bc219422aea5245666df7149ca3df2eca32122b085a" Jul 03 20:40:09.925391 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:09.925362 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f180df957689e22685ae7bc219422aea5245666df7149ca3df2eca32122b085a"} err="failed to get container status \"f180df957689e22685ae7bc219422aea5245666df7149ca3df2eca32122b085a\": rpc error: code = NotFound desc = could not find container \"f180df957689e22685ae7bc219422aea5245666df7149ca3df2eca32122b085a\": container with ID starting with f180df957689e22685ae7bc219422aea5245666df7149ca3df2eca32122b085a not found: ID does not exist" Jul 03 20:40:10.058571 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.058539 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-slzfp" Jul 03 20:40:10.073320 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.073297 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5cgc\" (UniqueName: \"kubernetes.io/projected/6678d1ce-f2bf-41fb-bdc4-047fd65c6030-kube-api-access-k5cgc\") pod \"6678d1ce-f2bf-41fb-bdc4-047fd65c6030\" (UID: \"6678d1ce-f2bf-41fb-bdc4-047fd65c6030\") " Jul 03 20:40:10.073431 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.073330 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6678d1ce-f2bf-41fb-bdc4-047fd65c6030-extensions-socket-volume\") pod \"6678d1ce-f2bf-41fb-bdc4-047fd65c6030\" (UID: \"6678d1ce-f2bf-41fb-bdc4-047fd65c6030\") " Jul 03 20:40:10.073431 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.073400 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlwd4\" (UniqueName: \"kubernetes.io/projected/56fec6b1-f650-4dbe-b6c8-a6263870181c-kube-api-access-zlwd4\") pod \"56fec6b1-f650-4dbe-b6c8-a6263870181c\" (UID: \"56fec6b1-f650-4dbe-b6c8-a6263870181c\") " Jul 03 20:40:10.074020 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.073983 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6678d1ce-f2bf-41fb-bdc4-047fd65c6030-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6678d1ce-f2bf-41fb-bdc4-047fd65c6030" (UID: "6678d1ce-f2bf-41fb-bdc4-047fd65c6030"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:40:10.075479 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.075446 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6678d1ce-f2bf-41fb-bdc4-047fd65c6030-kube-api-access-k5cgc" (OuterVolumeSpecName: "kube-api-access-k5cgc") pod "6678d1ce-f2bf-41fb-bdc4-047fd65c6030" (UID: "6678d1ce-f2bf-41fb-bdc4-047fd65c6030"). InnerVolumeSpecName "kube-api-access-k5cgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:40:10.075554 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.075507 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56fec6b1-f650-4dbe-b6c8-a6263870181c-kube-api-access-zlwd4" (OuterVolumeSpecName: "kube-api-access-zlwd4") pod "56fec6b1-f650-4dbe-b6c8-a6263870181c" (UID: "56fec6b1-f650-4dbe-b6c8-a6263870181c"). InnerVolumeSpecName "kube-api-access-zlwd4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:40:10.174446 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.174415 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5cgc\" (UniqueName: \"kubernetes.io/projected/6678d1ce-f2bf-41fb-bdc4-047fd65c6030-kube-api-access-k5cgc\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:40:10.174446 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.174440 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6678d1ce-f2bf-41fb-bdc4-047fd65c6030-extensions-socket-volume\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:40:10.174446 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.174449 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlwd4\" (UniqueName: \"kubernetes.io/projected/56fec6b1-f650-4dbe-b6c8-a6263870181c-kube-api-access-zlwd4\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:40:10.198147 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.198059 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-slzfp"] Jul 03 20:40:10.200399 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:40:10.200373 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a81c57_f8f4_4dd2_af7a_bc3e3ea143ec.slice/crio-29352d950fc8227a3ae16480a652eab9aeb52cf0619507fc44fa089bfd3b061e WatchSource:0}: Error finding container 29352d950fc8227a3ae16480a652eab9aeb52cf0619507fc44fa089bfd3b061e: Status 404 returned error can't find the container with id 29352d950fc8227a3ae16480a652eab9aeb52cf0619507fc44fa089bfd3b061e Jul 03 20:40:10.204577 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.204561 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" Jul 03 20:40:10.368204 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.368132 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh"] Jul 03 20:40:10.371279 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:40:10.371255 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa437708_a4ed_4bf5_a49d_8deab166366b.slice/crio-884adfa384e864a7275e3544046a1136a0b0deb9bbc49418ec36038df04d5351 WatchSource:0}: Error finding container 884adfa384e864a7275e3544046a1136a0b0deb9bbc49418ec36038df04d5351: Status 404 returned error can't find the container with id 884adfa384e864a7275e3544046a1136a0b0deb9bbc49418ec36038df04d5351 Jul 03 20:40:10.916626 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.916582 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-slzfp" event={"ID":"54a81c57-f8f4-4dd2-af7a-bc3e3ea143ec","Type":"ContainerStarted","Data":"831e7458f06ccaa4cbb411b4c54a985111f55fe6f316305e7511cb7f569c7331"} Jul 03 20:40:10.917074 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.916645 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-slzfp" event={"ID":"54a81c57-f8f4-4dd2-af7a-bc3e3ea143ec","Type":"ContainerStarted","Data":"29352d950fc8227a3ae16480a652eab9aeb52cf0619507fc44fa089bfd3b061e"} Jul 03 20:40:10.917074 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.916773 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-slzfp" Jul 03 20:40:10.919692 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.919661 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" event={"ID":"aa437708-a4ed-4bf5-a49d-8deab166366b","Type":"ContainerStarted","Data":"19034da65a42d189b8419768f4373e17833bb0660b0c348a723405d756975462"} Jul 03 20:40:10.919862 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.919702 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" event={"ID":"aa437708-a4ed-4bf5-a49d-8deab166366b","Type":"ContainerStarted","Data":"884adfa384e864a7275e3544046a1136a0b0deb9bbc49418ec36038df04d5351"} Jul 03 20:40:10.919862 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.919729 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" Jul 03 20:40:10.933424 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.933375 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-slzfp" podStartSLOduration=1.933360772 podStartE2EDuration="1.933360772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:40:10.932882706 +0000 UTC m=+477.391155145" watchObservedRunningTime="2026-07-03 20:40:10.933360772 +0000 UTC m=+477.391633213" Jul 03 20:40:10.951915 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:10.951867 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" podStartSLOduration=1.951851323 podStartE2EDuration="1.951851323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:40:10.951076178 +0000 UTC m=+477.409348617" watchObservedRunningTime="2026-07-03 20:40:10.951851323 +0000 UTC m=+477.410123761" Jul 03 20:40:11.985522 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:11.985490 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56fec6b1-f650-4dbe-b6c8-a6263870181c" path="/var/lib/kubelet/pods/56fec6b1-f650-4dbe-b6c8-a6263870181c/volumes" Jul 03 20:40:11.985905 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:11.985844 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6678d1ce-f2bf-41fb-bdc4-047fd65c6030" path="/var/lib/kubelet/pods/6678d1ce-f2bf-41fb-bdc4-047fd65c6030/volumes" Jul 03 20:40:21.925411 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:21.925382 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" Jul 03 20:40:21.925986 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:21.925464 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-slzfp" Jul 03 20:40:22.207095 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:22.207014 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk"] Jul 03 20:40:22.217414 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:22.217388 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk" Jul 03 20:40:22.226454 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:22.226421 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk"] Jul 03 20:40:22.259966 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:22.259929 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqd7c\" (UniqueName: \"kubernetes.io/projected/a4f476df-78c2-468b-9d44-0ecee9539e34-kube-api-access-rqd7c\") pod \"kuadrant-operator-controller-manager-86878c769f-s6pfk\" (UID: \"a4f476df-78c2-468b-9d44-0ecee9539e34\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk" Jul 03 20:40:22.259966 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:22.259968 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4f476df-78c2-468b-9d44-0ecee9539e34-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s6pfk\" (UID: \"a4f476df-78c2-468b-9d44-0ecee9539e34\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk" Jul 03 20:40:22.361084 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:22.361045 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqd7c\" (UniqueName: \"kubernetes.io/projected/a4f476df-78c2-468b-9d44-0ecee9539e34-kube-api-access-rqd7c\") pod \"kuadrant-operator-controller-manager-86878c769f-s6pfk\" (UID: \"a4f476df-78c2-468b-9d44-0ecee9539e34\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk" Jul 03 20:40:22.361263 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:22.361095 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4f476df-78c2-468b-9d44-0ecee9539e34-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s6pfk\" (UID: \"a4f476df-78c2-468b-9d44-0ecee9539e34\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk" Jul 03 20:40:22.361512 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:22.361495 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4f476df-78c2-468b-9d44-0ecee9539e34-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s6pfk\" (UID: \"a4f476df-78c2-468b-9d44-0ecee9539e34\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk" Jul 03 20:40:22.376900 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:22.376866 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqd7c\" (UniqueName: \"kubernetes.io/projected/a4f476df-78c2-468b-9d44-0ecee9539e34-kube-api-access-rqd7c\") pod \"kuadrant-operator-controller-manager-86878c769f-s6pfk\" (UID: \"a4f476df-78c2-468b-9d44-0ecee9539e34\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk" Jul 03 20:40:22.528722 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:22.528628 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk" Jul 03 20:40:22.664929 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:22.664853 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk"] Jul 03 20:40:22.669085 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:40:22.669060 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4f476df_78c2_468b_9d44_0ecee9539e34.slice/crio-6c293ce3de4b9fbbca70cee12e594e14423c93b14344574a65c63ccf3a315393 WatchSource:0}: Error finding container 6c293ce3de4b9fbbca70cee12e594e14423c93b14344574a65c63ccf3a315393: Status 404 returned error can't find the container with id 6c293ce3de4b9fbbca70cee12e594e14423c93b14344574a65c63ccf3a315393 Jul 03 20:40:22.974043 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:22.974005 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk" event={"ID":"a4f476df-78c2-468b-9d44-0ecee9539e34","Type":"ContainerStarted","Data":"9ca98eb548ea2eb1e60ab2442203725f890dd724e3a6a50c96086a149368c005"} Jul 03 20:40:22.974538 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:22.974061 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk" event={"ID":"a4f476df-78c2-468b-9d44-0ecee9539e34","Type":"ContainerStarted","Data":"6c293ce3de4b9fbbca70cee12e594e14423c93b14344574a65c63ccf3a315393"} Jul 03 20:40:22.974538 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:22.974165 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk" Jul 03 20:40:23.005035 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:23.004985 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk" podStartSLOduration=1.0049675 podStartE2EDuration="1.0049675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:40:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:40:23.001617448 +0000 UTC m=+489.459889887" watchObservedRunningTime="2026-07-03 20:40:23.0049675 +0000 UTC m=+489.463239938" Jul 03 20:40:33.985222 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:33.985193 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s6pfk" Jul 03 20:40:34.037190 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:34.037163 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh"] Jul 03 20:40:34.037415 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:34.037392 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" podUID="aa437708-a4ed-4bf5-a49d-8deab166366b" containerName="manager" containerID="cri-o://19034da65a42d189b8419768f4373e17833bb0660b0c348a723405d756975462" gracePeriod=10 Jul 03 20:40:34.283397 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:34.283373 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" Jul 03 20:40:34.349486 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:34.349456 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zhk9\" (UniqueName: \"kubernetes.io/projected/aa437708-a4ed-4bf5-a49d-8deab166366b-kube-api-access-5zhk9\") pod \"aa437708-a4ed-4bf5-a49d-8deab166366b\" (UID: \"aa437708-a4ed-4bf5-a49d-8deab166366b\") " Jul 03 20:40:34.349622 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:34.349520 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa437708-a4ed-4bf5-a49d-8deab166366b-extensions-socket-volume\") pod \"aa437708-a4ed-4bf5-a49d-8deab166366b\" (UID: \"aa437708-a4ed-4bf5-a49d-8deab166366b\") " Jul 03 20:40:34.349913 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:34.349889 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa437708-a4ed-4bf5-a49d-8deab166366b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "aa437708-a4ed-4bf5-a49d-8deab166366b" (UID: "aa437708-a4ed-4bf5-a49d-8deab166366b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:40:34.352269 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:34.352243 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa437708-a4ed-4bf5-a49d-8deab166366b-kube-api-access-5zhk9" (OuterVolumeSpecName: "kube-api-access-5zhk9") pod "aa437708-a4ed-4bf5-a49d-8deab166366b" (UID: "aa437708-a4ed-4bf5-a49d-8deab166366b"). InnerVolumeSpecName "kube-api-access-5zhk9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:40:34.450910 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:34.450878 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa437708-a4ed-4bf5-a49d-8deab166366b-extensions-socket-volume\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:40:34.450910 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:34.450904 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zhk9\" (UniqueName: \"kubernetes.io/projected/aa437708-a4ed-4bf5-a49d-8deab166366b-kube-api-access-5zhk9\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:40:35.026034 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:35.026000 2582 generic.go:356] "Generic (PLEG): container finished" podID="aa437708-a4ed-4bf5-a49d-8deab166366b" containerID="19034da65a42d189b8419768f4373e17833bb0660b0c348a723405d756975462" exitCode=0 Jul 03 20:40:35.026453 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:35.026042 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" Jul 03 20:40:35.026453 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:35.026094 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" event={"ID":"aa437708-a4ed-4bf5-a49d-8deab166366b","Type":"ContainerDied","Data":"19034da65a42d189b8419768f4373e17833bb0660b0c348a723405d756975462"} Jul 03 20:40:35.026453 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:35.026141 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh" event={"ID":"aa437708-a4ed-4bf5-a49d-8deab166366b","Type":"ContainerDied","Data":"884adfa384e864a7275e3544046a1136a0b0deb9bbc49418ec36038df04d5351"} Jul 03 20:40:35.026453 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:35.026163 2582 scope.go:117] "RemoveContainer" containerID="19034da65a42d189b8419768f4373e17833bb0660b0c348a723405d756975462" Jul 03 20:40:35.035058 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:35.035043 2582 scope.go:117] "RemoveContainer" containerID="19034da65a42d189b8419768f4373e17833bb0660b0c348a723405d756975462" Jul 03 20:40:35.035326 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:40:35.035309 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19034da65a42d189b8419768f4373e17833bb0660b0c348a723405d756975462\": container with ID starting with 19034da65a42d189b8419768f4373e17833bb0660b0c348a723405d756975462 not found: ID does not exist" containerID="19034da65a42d189b8419768f4373e17833bb0660b0c348a723405d756975462" Jul 03 20:40:35.035379 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:35.035336 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19034da65a42d189b8419768f4373e17833bb0660b0c348a723405d756975462"} err="failed to get container status \"19034da65a42d189b8419768f4373e17833bb0660b0c348a723405d756975462\": rpc error: code = NotFound desc = could not find container \"19034da65a42d189b8419768f4373e17833bb0660b0c348a723405d756975462\": container with ID starting with 19034da65a42d189b8419768f4373e17833bb0660b0c348a723405d756975462 not found: ID does not exist" Jul 03 20:40:35.047120 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:35.047099 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh"] Jul 03 20:40:35.054564 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:35.054543 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-mnlmh"] Jul 03 20:40:35.985481 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:35.985449 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa437708-a4ed-4bf5-a49d-8deab166366b" path="/var/lib/kubelet/pods/aa437708-a4ed-4bf5-a49d-8deab166366b/volumes" Jul 03 20:40:51.385471 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.385435 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zmgb2"] Jul 03 20:40:51.385862 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.385766 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa437708-a4ed-4bf5-a49d-8deab166366b" containerName="manager" Jul 03 20:40:51.385862 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.385777 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa437708-a4ed-4bf5-a49d-8deab166366b" containerName="manager" Jul 03 20:40:51.385862 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.385853 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa437708-a4ed-4bf5-a49d-8deab166366b" containerName="manager" Jul 03 20:40:51.388958 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.388942 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" Jul 03 20:40:51.393067 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.393043 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 03 20:40:51.393201 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.393142 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-q6cg7\"" Jul 03 20:40:51.402288 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.402265 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zmgb2"] Jul 03 20:40:51.480353 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.480326 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zmgb2"] Jul 03 20:40:51.483710 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.483685 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db4mb\" (UniqueName: \"kubernetes.io/projected/a5cb381b-3031-4da2-bd21-20b93d5a8fa2-kube-api-access-db4mb\") pod \"limitador-limitador-64c94fd64f-zmgb2\" (UID: \"a5cb381b-3031-4da2-bd21-20b93d5a8fa2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" Jul 03 20:40:51.483880 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.483741 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a5cb381b-3031-4da2-bd21-20b93d5a8fa2-config-file\") pod \"limitador-limitador-64c94fd64f-zmgb2\" (UID: \"a5cb381b-3031-4da2-bd21-20b93d5a8fa2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" Jul 03 20:40:51.584522 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.584481 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db4mb\" (UniqueName: \"kubernetes.io/projected/a5cb381b-3031-4da2-bd21-20b93d5a8fa2-kube-api-access-db4mb\") pod \"limitador-limitador-64c94fd64f-zmgb2\" (UID: \"a5cb381b-3031-4da2-bd21-20b93d5a8fa2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" Jul 03 20:40:51.584692 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.584559 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a5cb381b-3031-4da2-bd21-20b93d5a8fa2-config-file\") pod \"limitador-limitador-64c94fd64f-zmgb2\" (UID: \"a5cb381b-3031-4da2-bd21-20b93d5a8fa2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" Jul 03 20:40:51.585245 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.585224 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a5cb381b-3031-4da2-bd21-20b93d5a8fa2-config-file\") pod \"limitador-limitador-64c94fd64f-zmgb2\" (UID: \"a5cb381b-3031-4da2-bd21-20b93d5a8fa2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" Jul 03 20:40:51.595416 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.595388 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-db4mb\" (UniqueName: \"kubernetes.io/projected/a5cb381b-3031-4da2-bd21-20b93d5a8fa2-kube-api-access-db4mb\") pod \"limitador-limitador-64c94fd64f-zmgb2\" (UID: \"a5cb381b-3031-4da2-bd21-20b93d5a8fa2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" Jul 03 20:40:51.703107 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.703074 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" Jul 03 20:40:51.834929 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:51.834897 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zmgb2"] Jul 03 20:40:51.838062 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:40:51.838035 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5cb381b_3031_4da2_bd21_20b93d5a8fa2.slice/crio-c985392d19f89a461a95dde9f4db0f746f079722610d26a1a1b77d18de4de828 WatchSource:0}: Error finding container c985392d19f89a461a95dde9f4db0f746f079722610d26a1a1b77d18de4de828: Status 404 returned error can't find the container with id c985392d19f89a461a95dde9f4db0f746f079722610d26a1a1b77d18de4de828 Jul 03 20:40:52.102506 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:52.102423 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" event={"ID":"a5cb381b-3031-4da2-bd21-20b93d5a8fa2","Type":"ContainerStarted","Data":"c985392d19f89a461a95dde9f4db0f746f079722610d26a1a1b77d18de4de828"} Jul 03 20:40:52.318055 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:52.318021 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-26x95"] Jul 03 20:40:52.322432 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:52.322411 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-26x95" Jul 03 20:40:52.325003 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:52.324982 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d5t8k\"" Jul 03 20:40:52.328871 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:52.328852 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-26x95"] Jul 03 20:40:52.391535 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:52.391461 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75rvd\" (UniqueName: \"kubernetes.io/projected/3935174b-d229-4aff-b905-77f42068a7fa-kube-api-access-75rvd\") pod \"authorino-75d556bd77-26x95\" (UID: \"3935174b-d229-4aff-b905-77f42068a7fa\") " pod="kuadrant-system/authorino-75d556bd77-26x95" Jul 03 20:40:52.492077 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:52.492039 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75rvd\" (UniqueName: \"kubernetes.io/projected/3935174b-d229-4aff-b905-77f42068a7fa-kube-api-access-75rvd\") pod \"authorino-75d556bd77-26x95\" (UID: \"3935174b-d229-4aff-b905-77f42068a7fa\") " pod="kuadrant-system/authorino-75d556bd77-26x95" Jul 03 20:40:52.507191 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:52.507163 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75rvd\" (UniqueName: \"kubernetes.io/projected/3935174b-d229-4aff-b905-77f42068a7fa-kube-api-access-75rvd\") pod \"authorino-75d556bd77-26x95\" (UID: \"3935174b-d229-4aff-b905-77f42068a7fa\") " pod="kuadrant-system/authorino-75d556bd77-26x95" Jul 03 20:40:52.632728 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:52.632549 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-26x95" Jul 03 20:40:52.806486 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:52.806453 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-26x95"] Jul 03 20:40:52.810120 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:40:52.810080 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3935174b_d229_4aff_b905_77f42068a7fa.slice/crio-da73b9e99ac75a2cee82c8fa4d0739a8d000c64b567ff7096639cf265452bd16 WatchSource:0}: Error finding container da73b9e99ac75a2cee82c8fa4d0739a8d000c64b567ff7096639cf265452bd16: Status 404 returned error can't find the container with id da73b9e99ac75a2cee82c8fa4d0739a8d000c64b567ff7096639cf265452bd16 Jul 03 20:40:53.108552 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:53.108513 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-26x95" event={"ID":"3935174b-d229-4aff-b905-77f42068a7fa","Type":"ContainerStarted","Data":"da73b9e99ac75a2cee82c8fa4d0739a8d000c64b567ff7096639cf265452bd16"} Jul 03 20:40:57.127723 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:57.127688 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" event={"ID":"a5cb381b-3031-4da2-bd21-20b93d5a8fa2","Type":"ContainerStarted","Data":"ab8022e6c55ac39bf6f8da8eab0ae390b5c796b4b135d41b5dfd6a9e19c62b9c"} Jul 03 20:40:57.128196 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:57.127817 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" Jul 03 20:40:57.129288 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:57.129268 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-26x95" event={"ID":"3935174b-d229-4aff-b905-77f42068a7fa","Type":"ContainerStarted","Data":"4571045a23d28591e814e7f661ee2ec5082af6c661df63e6cd0a773fabc151ed"} Jul 03 20:40:57.144101 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:57.144023 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" podStartSLOduration=1.522698484 podStartE2EDuration="6.144012007s" totalImagesPullingTime="4.621313523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:40:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:40:57.142843299 +0000 UTC m=+523.601115739" watchObservedRunningTime="2026-07-03 20:40:57.144012007 +0000 UTC m=+523.602284446" Jul 03 20:40:57.157361 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:40:57.157321 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-26x95" podStartSLOduration=1.4970156829999999 podStartE2EDuration="5.157310186s" totalImagesPullingTime="3.660294503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:40:57.156126886 +0000 UTC m=+523.614399329" watchObservedRunningTime="2026-07-03 20:40:57.157310186 +0000 UTC m=+523.615582640" Jul 03 20:41:08.134071 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:08.133995 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" Jul 03 20:41:08.389891 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:08.389818 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zmgb2"] Jul 03 20:41:08.390025 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:08.389990 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" podUID="a5cb381b-3031-4da2-bd21-20b93d5a8fa2" containerName="limitador" containerID="cri-o://ab8022e6c55ac39bf6f8da8eab0ae390b5c796b4b135d41b5dfd6a9e19c62b9c" gracePeriod=30 Jul 03 20:41:08.934016 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:08.933989 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" Jul 03 20:41:09.029699 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.029623 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a5cb381b-3031-4da2-bd21-20b93d5a8fa2-config-file\") pod \"a5cb381b-3031-4da2-bd21-20b93d5a8fa2\" (UID: \"a5cb381b-3031-4da2-bd21-20b93d5a8fa2\") " Jul 03 20:41:09.029699 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.029664 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db4mb\" (UniqueName: \"kubernetes.io/projected/a5cb381b-3031-4da2-bd21-20b93d5a8fa2-kube-api-access-db4mb\") pod \"a5cb381b-3031-4da2-bd21-20b93d5a8fa2\" (UID: \"a5cb381b-3031-4da2-bd21-20b93d5a8fa2\") " Jul 03 20:41:09.032834 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.030260 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5cb381b-3031-4da2-bd21-20b93d5a8fa2-config-file" (OuterVolumeSpecName: "config-file") pod "a5cb381b-3031-4da2-bd21-20b93d5a8fa2" (UID: "a5cb381b-3031-4da2-bd21-20b93d5a8fa2"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:41:09.036183 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.036148 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5cb381b-3031-4da2-bd21-20b93d5a8fa2-kube-api-access-db4mb" (OuterVolumeSpecName: "kube-api-access-db4mb") pod "a5cb381b-3031-4da2-bd21-20b93d5a8fa2" (UID: "a5cb381b-3031-4da2-bd21-20b93d5a8fa2"). InnerVolumeSpecName "kube-api-access-db4mb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:41:09.130475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.130442 2582 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a5cb381b-3031-4da2-bd21-20b93d5a8fa2-config-file\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:41:09.130475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.130471 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db4mb\" (UniqueName: \"kubernetes.io/projected/a5cb381b-3031-4da2-bd21-20b93d5a8fa2-kube-api-access-db4mb\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:41:09.179448 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.179414 2582 generic.go:356] "Generic (PLEG): container finished" podID="a5cb381b-3031-4da2-bd21-20b93d5a8fa2" containerID="ab8022e6c55ac39bf6f8da8eab0ae390b5c796b4b135d41b5dfd6a9e19c62b9c" exitCode=0 Jul 03 20:41:09.179803 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.179467 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" Jul 03 20:41:09.179803 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.179492 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" event={"ID":"a5cb381b-3031-4da2-bd21-20b93d5a8fa2","Type":"ContainerDied","Data":"ab8022e6c55ac39bf6f8da8eab0ae390b5c796b4b135d41b5dfd6a9e19c62b9c"} Jul 03 20:41:09.179803 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.179555 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zmgb2" event={"ID":"a5cb381b-3031-4da2-bd21-20b93d5a8fa2","Type":"ContainerDied","Data":"c985392d19f89a461a95dde9f4db0f746f079722610d26a1a1b77d18de4de828"} Jul 03 20:41:09.179803 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.179580 2582 scope.go:117] "RemoveContainer" containerID="ab8022e6c55ac39bf6f8da8eab0ae390b5c796b4b135d41b5dfd6a9e19c62b9c" Jul 03 20:41:09.187991 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.187973 2582 scope.go:117] "RemoveContainer" containerID="ab8022e6c55ac39bf6f8da8eab0ae390b5c796b4b135d41b5dfd6a9e19c62b9c" Jul 03 20:41:09.188267 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:41:09.188250 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab8022e6c55ac39bf6f8da8eab0ae390b5c796b4b135d41b5dfd6a9e19c62b9c\": container with ID starting with ab8022e6c55ac39bf6f8da8eab0ae390b5c796b4b135d41b5dfd6a9e19c62b9c not found: ID does not exist" containerID="ab8022e6c55ac39bf6f8da8eab0ae390b5c796b4b135d41b5dfd6a9e19c62b9c" Jul 03 20:41:09.188305 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.188274 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab8022e6c55ac39bf6f8da8eab0ae390b5c796b4b135d41b5dfd6a9e19c62b9c"} err="failed to get container status \"ab8022e6c55ac39bf6f8da8eab0ae390b5c796b4b135d41b5dfd6a9e19c62b9c\": rpc error: code = NotFound desc = could not find container \"ab8022e6c55ac39bf6f8da8eab0ae390b5c796b4b135d41b5dfd6a9e19c62b9c\": container with ID starting with ab8022e6c55ac39bf6f8da8eab0ae390b5c796b4b135d41b5dfd6a9e19c62b9c not found: ID does not exist" Jul 03 20:41:09.199549 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.199529 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zmgb2"] Jul 03 20:41:09.205386 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.205361 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zmgb2"] Jul 03 20:41:09.985210 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:09.985176 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5cb381b-3031-4da2-bd21-20b93d5a8fa2" path="/var/lib/kubelet/pods/a5cb381b-3031-4da2-bd21-20b93d5a8fa2/volumes" Jul 03 20:41:12.572927 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.572892 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-8xzxp"] Jul 03 20:41:12.573371 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.573355 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5cb381b-3031-4da2-bd21-20b93d5a8fa2" containerName="limitador" Jul 03 20:41:12.573412 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.573374 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5cb381b-3031-4da2-bd21-20b93d5a8fa2" containerName="limitador" Jul 03 20:41:12.573483 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.573472 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5cb381b-3031-4da2-bd21-20b93d5a8fa2" containerName="limitador" Jul 03 20:41:12.578090 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.578070 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-8xzxp" Jul 03 20:41:12.580540 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.580520 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 03 20:41:12.580684 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.580609 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tl4mm\"" Jul 03 20:41:12.582565 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.582546 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-8xzxp"] Jul 03 20:41:12.658485 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.658454 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckn75\" (UniqueName: \"kubernetes.io/projected/21b1004f-1493-4912-b664-73588801f758-kube-api-access-ckn75\") pod \"postgres-69fb4594cf-8xzxp\" (UID: \"21b1004f-1493-4912-b664-73588801f758\") " pod="opendatahub/postgres-69fb4594cf-8xzxp" Jul 03 20:41:12.658670 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.658525 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/21b1004f-1493-4912-b664-73588801f758-data\") pod \"postgres-69fb4594cf-8xzxp\" (UID: \"21b1004f-1493-4912-b664-73588801f758\") " pod="opendatahub/postgres-69fb4594cf-8xzxp" Jul 03 20:41:12.759825 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.759784 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckn75\" (UniqueName: \"kubernetes.io/projected/21b1004f-1493-4912-b664-73588801f758-kube-api-access-ckn75\") pod \"postgres-69fb4594cf-8xzxp\" (UID: \"21b1004f-1493-4912-b664-73588801f758\") " pod="opendatahub/postgres-69fb4594cf-8xzxp" Jul 03 20:41:12.760018 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.759890 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/21b1004f-1493-4912-b664-73588801f758-data\") pod \"postgres-69fb4594cf-8xzxp\" (UID: \"21b1004f-1493-4912-b664-73588801f758\") " pod="opendatahub/postgres-69fb4594cf-8xzxp" Jul 03 20:41:12.760269 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.760249 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/21b1004f-1493-4912-b664-73588801f758-data\") pod \"postgres-69fb4594cf-8xzxp\" (UID: \"21b1004f-1493-4912-b664-73588801f758\") " pod="opendatahub/postgres-69fb4594cf-8xzxp" Jul 03 20:41:12.769713 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.769690 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckn75\" (UniqueName: \"kubernetes.io/projected/21b1004f-1493-4912-b664-73588801f758-kube-api-access-ckn75\") pod \"postgres-69fb4594cf-8xzxp\" (UID: \"21b1004f-1493-4912-b664-73588801f758\") " pod="opendatahub/postgres-69fb4594cf-8xzxp" Jul 03 20:41:12.890078 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:12.889986 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-8xzxp" Jul 03 20:41:13.020437 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:13.020414 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-8xzxp"] Jul 03 20:41:13.022496 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:41:13.022474 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b1004f_1493_4912_b664_73588801f758.slice/crio-d8841e344aa8c5df9aa16bbc58424caf9a96e8e8d306df6737a0cf12144cbaa8 WatchSource:0}: Error finding container d8841e344aa8c5df9aa16bbc58424caf9a96e8e8d306df6737a0cf12144cbaa8: Status 404 returned error can't find the container with id d8841e344aa8c5df9aa16bbc58424caf9a96e8e8d306df6737a0cf12144cbaa8 Jul 03 20:41:13.198607 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:13.198570 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-8xzxp" event={"ID":"21b1004f-1493-4912-b664-73588801f758","Type":"ContainerStarted","Data":"d8841e344aa8c5df9aa16bbc58424caf9a96e8e8d306df6737a0cf12144cbaa8"} Jul 03 20:41:17.847847 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:17.847821 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 03 20:41:18.224292 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:18.224257 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-8xzxp" event={"ID":"21b1004f-1493-4912-b664-73588801f758","Type":"ContainerStarted","Data":"9c298f28558c5116e8b6d2ff10fdb99e2b68ec6fa50b34fa80ba4c35b91934fb"} Jul 03 20:41:18.224457 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:18.224407 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-8xzxp" Jul 03 20:41:18.244366 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:18.244319 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-8xzxp" podStartSLOduration=1.423077628 podStartE2EDuration="6.244304739s" totalImagesPullingTime="4.821227111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:41:18.243074607 +0000 UTC m=+544.701347047" watchObservedRunningTime="2026-07-03 20:41:18.244304739 +0000 UTC m=+544.702577177" Jul 03 20:41:24.265400 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:24.265371 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-8xzxp" Jul 03 20:41:24.874688 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:24.874655 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8xs88"] Jul 03 20:41:24.881285 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:24.881260 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8xs88" Jul 03 20:41:24.884827 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:24.884782 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8xs88"] Jul 03 20:41:25.067113 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.067081 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86pr2\" (UniqueName: \"kubernetes.io/projected/59ce679f-9360-4277-a280-9ba2fe26dba4-kube-api-access-86pr2\") pod \"authorino-5d4b67c446-8xs88\" (UID: \"59ce679f-9360-4277-a280-9ba2fe26dba4\") " pod="kuadrant-system/authorino-5d4b67c446-8xs88" Jul 03 20:41:25.131108 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.131015 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8xs88"] Jul 03 20:41:25.131321 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:41:25.131298 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-86pr2], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-8xs88" podUID="59ce679f-9360-4277-a280-9ba2fe26dba4" Jul 03 20:41:25.156054 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.156013 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-794ff96965-m9vfv"] Jul 03 20:41:25.159471 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.159449 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-794ff96965-m9vfv" Jul 03 20:41:25.165318 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.165294 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-794ff96965-m9vfv"] Jul 03 20:41:25.168083 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.168059 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86pr2\" (UniqueName: \"kubernetes.io/projected/59ce679f-9360-4277-a280-9ba2fe26dba4-kube-api-access-86pr2\") pod \"authorino-5d4b67c446-8xs88\" (UID: \"59ce679f-9360-4277-a280-9ba2fe26dba4\") " pod="kuadrant-system/authorino-5d4b67c446-8xs88" Jul 03 20:41:25.180548 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.180511 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86pr2\" (UniqueName: \"kubernetes.io/projected/59ce679f-9360-4277-a280-9ba2fe26dba4-kube-api-access-86pr2\") pod \"authorino-5d4b67c446-8xs88\" (UID: \"59ce679f-9360-4277-a280-9ba2fe26dba4\") " pod="kuadrant-system/authorino-5d4b67c446-8xs88" Jul 03 20:41:25.261274 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.261243 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8xs88" Jul 03 20:41:25.268467 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.268444 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8xs88" Jul 03 20:41:25.268820 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.268806 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxtxl\" (UniqueName: \"kubernetes.io/projected/c0e5224e-5b53-4e41-9f1b-c942dc08b3e3-kube-api-access-pxtxl\") pod \"authorino-794ff96965-m9vfv\" (UID: \"c0e5224e-5b53-4e41-9f1b-c942dc08b3e3\") " pod="kuadrant-system/authorino-794ff96965-m9vfv" Jul 03 20:41:25.369497 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.369461 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86pr2\" (UniqueName: \"kubernetes.io/projected/59ce679f-9360-4277-a280-9ba2fe26dba4-kube-api-access-86pr2\") pod \"59ce679f-9360-4277-a280-9ba2fe26dba4\" (UID: \"59ce679f-9360-4277-a280-9ba2fe26dba4\") " Jul 03 20:41:25.369649 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.369612 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxtxl\" (UniqueName: \"kubernetes.io/projected/c0e5224e-5b53-4e41-9f1b-c942dc08b3e3-kube-api-access-pxtxl\") pod \"authorino-794ff96965-m9vfv\" (UID: \"c0e5224e-5b53-4e41-9f1b-c942dc08b3e3\") " pod="kuadrant-system/authorino-794ff96965-m9vfv" Jul 03 20:41:25.371708 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.371679 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59ce679f-9360-4277-a280-9ba2fe26dba4-kube-api-access-86pr2" (OuterVolumeSpecName: "kube-api-access-86pr2") pod "59ce679f-9360-4277-a280-9ba2fe26dba4" (UID: "59ce679f-9360-4277-a280-9ba2fe26dba4"). InnerVolumeSpecName "kube-api-access-86pr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:41:25.378123 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.378105 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxtxl\" (UniqueName: \"kubernetes.io/projected/c0e5224e-5b53-4e41-9f1b-c942dc08b3e3-kube-api-access-pxtxl\") pod \"authorino-794ff96965-m9vfv\" (UID: \"c0e5224e-5b53-4e41-9f1b-c942dc08b3e3\") " pod="kuadrant-system/authorino-794ff96965-m9vfv" Jul 03 20:41:25.407182 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.407157 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-794ff96965-m9vfv"] Jul 03 20:41:25.407386 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.407376 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-794ff96965-m9vfv" Jul 03 20:41:25.438157 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.438130 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65bcd68d89-jbsh9"] Jul 03 20:41:25.442607 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.442586 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65bcd68d89-jbsh9" Jul 03 20:41:25.445129 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.445111 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 03 20:41:25.448967 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.448945 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65bcd68d89-jbsh9"] Jul 03 20:41:25.471199 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.471079 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86pr2\" (UniqueName: \"kubernetes.io/projected/59ce679f-9360-4277-a280-9ba2fe26dba4-kube-api-access-86pr2\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:41:25.546474 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.546436 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-794ff96965-m9vfv"] Jul 03 20:41:25.548787 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:41:25.548764 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0e5224e_5b53_4e41_9f1b_c942dc08b3e3.slice/crio-76821875f158a7c192bb2163717191c4e968cedd62a50fb4211784dcd86f3fb3 WatchSource:0}: Error finding container 76821875f158a7c192bb2163717191c4e968cedd62a50fb4211784dcd86f3fb3: Status 404 returned error can't find the container with id 76821875f158a7c192bb2163717191c4e968cedd62a50fb4211784dcd86f3fb3 Jul 03 20:41:25.572121 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.572091 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/51811eed-c619-430a-8fc7-28dc31177876-tls-cert\") pod \"authorino-65bcd68d89-jbsh9\" (UID: \"51811eed-c619-430a-8fc7-28dc31177876\") " pod="kuadrant-system/authorino-65bcd68d89-jbsh9" Jul 03 20:41:25.572257 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.572209 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7hwh\" (UniqueName: \"kubernetes.io/projected/51811eed-c619-430a-8fc7-28dc31177876-kube-api-access-d7hwh\") pod \"authorino-65bcd68d89-jbsh9\" (UID: \"51811eed-c619-430a-8fc7-28dc31177876\") " pod="kuadrant-system/authorino-65bcd68d89-jbsh9" Jul 03 20:41:25.673522 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.673436 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7hwh\" (UniqueName: \"kubernetes.io/projected/51811eed-c619-430a-8fc7-28dc31177876-kube-api-access-d7hwh\") pod \"authorino-65bcd68d89-jbsh9\" (UID: \"51811eed-c619-430a-8fc7-28dc31177876\") " pod="kuadrant-system/authorino-65bcd68d89-jbsh9" Jul 03 20:41:25.673522 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.673489 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/51811eed-c619-430a-8fc7-28dc31177876-tls-cert\") pod \"authorino-65bcd68d89-jbsh9\" (UID: \"51811eed-c619-430a-8fc7-28dc31177876\") " pod="kuadrant-system/authorino-65bcd68d89-jbsh9" Jul 03 20:41:25.676029 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.676001 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/51811eed-c619-430a-8fc7-28dc31177876-tls-cert\") pod \"authorino-65bcd68d89-jbsh9\" (UID: \"51811eed-c619-430a-8fc7-28dc31177876\") " pod="kuadrant-system/authorino-65bcd68d89-jbsh9" Jul 03 20:41:25.682376 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.682351 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7hwh\" (UniqueName: \"kubernetes.io/projected/51811eed-c619-430a-8fc7-28dc31177876-kube-api-access-d7hwh\") pod \"authorino-65bcd68d89-jbsh9\" (UID: \"51811eed-c619-430a-8fc7-28dc31177876\") " pod="kuadrant-system/authorino-65bcd68d89-jbsh9" Jul 03 20:41:25.754282 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.754249 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65bcd68d89-jbsh9" Jul 03 20:41:25.886695 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:25.886666 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65bcd68d89-jbsh9"] Jul 03 20:41:25.889979 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:41:25.889951 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51811eed_c619_430a_8fc7_28dc31177876.slice/crio-c5e3b698dcd445f60013e2140944998e07955bc2c07e13b776136fc0729d41e9 WatchSource:0}: Error finding container c5e3b698dcd445f60013e2140944998e07955bc2c07e13b776136fc0729d41e9: Status 404 returned error can't find the container with id c5e3b698dcd445f60013e2140944998e07955bc2c07e13b776136fc0729d41e9 Jul 03 20:41:26.267177 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:26.267135 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65bcd68d89-jbsh9" event={"ID":"51811eed-c619-430a-8fc7-28dc31177876","Type":"ContainerStarted","Data":"c5e3b698dcd445f60013e2140944998e07955bc2c07e13b776136fc0729d41e9"} Jul 03 20:41:26.268979 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:26.268958 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8xs88" Jul 03 20:41:26.269301 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:26.268963 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-794ff96965-m9vfv" podUID="c0e5224e-5b53-4e41-9f1b-c942dc08b3e3" containerName="authorino" containerID="cri-o://fb24623546416ecc2a43997ec892d6d512a1624167c859ad4226690922f7c19f" gracePeriod=30 Jul 03 20:41:26.269301 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:26.269000 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-794ff96965-m9vfv" event={"ID":"c0e5224e-5b53-4e41-9f1b-c942dc08b3e3","Type":"ContainerStarted","Data":"fb24623546416ecc2a43997ec892d6d512a1624167c859ad4226690922f7c19f"} Jul 03 20:41:26.269301 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:26.269038 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-794ff96965-m9vfv" event={"ID":"c0e5224e-5b53-4e41-9f1b-c942dc08b3e3","Type":"ContainerStarted","Data":"76821875f158a7c192bb2163717191c4e968cedd62a50fb4211784dcd86f3fb3"} Jul 03 20:41:26.286857 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:26.286817 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-794ff96965-m9vfv" podStartSLOduration=0.856897752 podStartE2EDuration="1.286803529s" totalImagesPullingTime="429.905777ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:41:26.285676328 +0000 UTC m=+552.743948780" watchObservedRunningTime="2026-07-03 20:41:26.286803529 +0000 UTC m=+552.745075968" Jul 03 20:41:26.308099 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:26.308066 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8xs88"] Jul 03 20:41:26.311524 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:26.311499 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8xs88"] Jul 03 20:41:26.521020 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:26.520998 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-794ff96965-m9vfv" Jul 03 20:41:26.680516 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:26.680487 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxtxl\" (UniqueName: \"kubernetes.io/projected/c0e5224e-5b53-4e41-9f1b-c942dc08b3e3-kube-api-access-pxtxl\") pod \"c0e5224e-5b53-4e41-9f1b-c942dc08b3e3\" (UID: \"c0e5224e-5b53-4e41-9f1b-c942dc08b3e3\") " Jul 03 20:41:26.682823 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:26.682786 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0e5224e-5b53-4e41-9f1b-c942dc08b3e3-kube-api-access-pxtxl" (OuterVolumeSpecName: "kube-api-access-pxtxl") pod "c0e5224e-5b53-4e41-9f1b-c942dc08b3e3" (UID: "c0e5224e-5b53-4e41-9f1b-c942dc08b3e3"). InnerVolumeSpecName "kube-api-access-pxtxl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:41:26.781360 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:26.781279 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxtxl\" (UniqueName: \"kubernetes.io/projected/c0e5224e-5b53-4e41-9f1b-c942dc08b3e3-kube-api-access-pxtxl\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:41:27.274986 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.274949 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65bcd68d89-jbsh9" event={"ID":"51811eed-c619-430a-8fc7-28dc31177876","Type":"ContainerStarted","Data":"7a43ed4ac020792437592dfe6b30a9de3fbd73bb38514d626998fc005c48bd89"} Jul 03 20:41:27.276144 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.276121 2582 generic.go:356] "Generic (PLEG): container finished" podID="c0e5224e-5b53-4e41-9f1b-c942dc08b3e3" containerID="fb24623546416ecc2a43997ec892d6d512a1624167c859ad4226690922f7c19f" exitCode=0 Jul 03 20:41:27.276238 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.276167 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-794ff96965-m9vfv" Jul 03 20:41:27.276238 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.276184 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-794ff96965-m9vfv" event={"ID":"c0e5224e-5b53-4e41-9f1b-c942dc08b3e3","Type":"ContainerDied","Data":"fb24623546416ecc2a43997ec892d6d512a1624167c859ad4226690922f7c19f"} Jul 03 20:41:27.276238 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.276209 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-794ff96965-m9vfv" event={"ID":"c0e5224e-5b53-4e41-9f1b-c942dc08b3e3","Type":"ContainerDied","Data":"76821875f158a7c192bb2163717191c4e968cedd62a50fb4211784dcd86f3fb3"} Jul 03 20:41:27.276238 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.276225 2582 scope.go:117] "RemoveContainer" containerID="fb24623546416ecc2a43997ec892d6d512a1624167c859ad4226690922f7c19f" Jul 03 20:41:27.285145 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.285127 2582 scope.go:117] "RemoveContainer" containerID="fb24623546416ecc2a43997ec892d6d512a1624167c859ad4226690922f7c19f" Jul 03 20:41:27.285429 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:41:27.285414 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb24623546416ecc2a43997ec892d6d512a1624167c859ad4226690922f7c19f\": container with ID starting with fb24623546416ecc2a43997ec892d6d512a1624167c859ad4226690922f7c19f not found: ID does not exist" containerID="fb24623546416ecc2a43997ec892d6d512a1624167c859ad4226690922f7c19f" Jul 03 20:41:27.285476 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.285438 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb24623546416ecc2a43997ec892d6d512a1624167c859ad4226690922f7c19f"} err="failed to get container status \"fb24623546416ecc2a43997ec892d6d512a1624167c859ad4226690922f7c19f\": rpc error: code = NotFound desc = could not find container \"fb24623546416ecc2a43997ec892d6d512a1624167c859ad4226690922f7c19f\": container with ID starting with fb24623546416ecc2a43997ec892d6d512a1624167c859ad4226690922f7c19f not found: ID does not exist" Jul 03 20:41:27.296313 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.296267 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-65bcd68d89-jbsh9" podStartSLOduration=1.984236764 podStartE2EDuration="2.296252464s" totalImagesPullingTime="312.0157ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:41:27.295932305 +0000 UTC m=+553.754204746" watchObservedRunningTime="2026-07-03 20:41:27.296252464 +0000 UTC m=+553.754524905" Jul 03 20:41:27.310788 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.310740 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-794ff96965-m9vfv"] Jul 03 20:41:27.314512 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.314488 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-794ff96965-m9vfv"] Jul 03 20:41:27.329540 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.329513 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-26x95"] Jul 03 20:41:27.330042 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.329992 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-26x95" podUID="3935174b-d229-4aff-b905-77f42068a7fa" containerName="authorino" containerID="cri-o://4571045a23d28591e814e7f661ee2ec5082af6c661df63e6cd0a773fabc151ed" gracePeriod=30 Jul 03 20:41:27.593481 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.593439 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-26x95" Jul 03 20:41:27.788352 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.788264 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75rvd\" (UniqueName: \"kubernetes.io/projected/3935174b-d229-4aff-b905-77f42068a7fa-kube-api-access-75rvd\") pod \"3935174b-d229-4aff-b905-77f42068a7fa\" (UID: \"3935174b-d229-4aff-b905-77f42068a7fa\") " Jul 03 20:41:27.790430 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.790401 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3935174b-d229-4aff-b905-77f42068a7fa-kube-api-access-75rvd" (OuterVolumeSpecName: "kube-api-access-75rvd") pod "3935174b-d229-4aff-b905-77f42068a7fa" (UID: "3935174b-d229-4aff-b905-77f42068a7fa"). InnerVolumeSpecName "kube-api-access-75rvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:41:27.889061 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.889027 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75rvd\" (UniqueName: \"kubernetes.io/projected/3935174b-d229-4aff-b905-77f42068a7fa-kube-api-access-75rvd\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:41:27.991982 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.991941 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59ce679f-9360-4277-a280-9ba2fe26dba4" path="/var/lib/kubelet/pods/59ce679f-9360-4277-a280-9ba2fe26dba4/volumes" Jul 03 20:41:27.992228 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:27.992211 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0e5224e-5b53-4e41-9f1b-c942dc08b3e3" path="/var/lib/kubelet/pods/c0e5224e-5b53-4e41-9f1b-c942dc08b3e3/volumes" Jul 03 20:41:28.282715 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:28.282681 2582 generic.go:356] "Generic (PLEG): container finished" podID="3935174b-d229-4aff-b905-77f42068a7fa" containerID="4571045a23d28591e814e7f661ee2ec5082af6c661df63e6cd0a773fabc151ed" exitCode=0 Jul 03 20:41:28.283173 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:28.282735 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-26x95" Jul 03 20:41:28.283173 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:28.282774 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-26x95" event={"ID":"3935174b-d229-4aff-b905-77f42068a7fa","Type":"ContainerDied","Data":"4571045a23d28591e814e7f661ee2ec5082af6c661df63e6cd0a773fabc151ed"} Jul 03 20:41:28.283173 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:28.282818 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-26x95" event={"ID":"3935174b-d229-4aff-b905-77f42068a7fa","Type":"ContainerDied","Data":"da73b9e99ac75a2cee82c8fa4d0739a8d000c64b567ff7096639cf265452bd16"} Jul 03 20:41:28.283173 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:28.282841 2582 scope.go:117] "RemoveContainer" containerID="4571045a23d28591e814e7f661ee2ec5082af6c661df63e6cd0a773fabc151ed" Jul 03 20:41:28.290962 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:28.290942 2582 scope.go:117] "RemoveContainer" containerID="4571045a23d28591e814e7f661ee2ec5082af6c661df63e6cd0a773fabc151ed" Jul 03 20:41:28.291185 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:41:28.291167 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4571045a23d28591e814e7f661ee2ec5082af6c661df63e6cd0a773fabc151ed\": container with ID starting with 4571045a23d28591e814e7f661ee2ec5082af6c661df63e6cd0a773fabc151ed not found: ID does not exist" containerID="4571045a23d28591e814e7f661ee2ec5082af6c661df63e6cd0a773fabc151ed" Jul 03 20:41:28.291231 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:28.291194 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4571045a23d28591e814e7f661ee2ec5082af6c661df63e6cd0a773fabc151ed"} err="failed to get container status \"4571045a23d28591e814e7f661ee2ec5082af6c661df63e6cd0a773fabc151ed\": rpc error: code = NotFound desc = could not find container \"4571045a23d28591e814e7f661ee2ec5082af6c661df63e6cd0a773fabc151ed\": container with ID starting with 4571045a23d28591e814e7f661ee2ec5082af6c661df63e6cd0a773fabc151ed not found: ID does not exist" Jul 03 20:41:28.298429 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:28.298409 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-26x95"] Jul 03 20:41:28.302591 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:28.302572 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-26x95"] Jul 03 20:41:29.985594 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:29.985562 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3935174b-d229-4aff-b905-77f42068a7fa" path="/var/lib/kubelet/pods/3935174b-d229-4aff-b905-77f42068a7fa/volumes" Jul 03 20:41:32.023320 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.023293 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5776c5ff6-kl9l2"] Jul 03 20:41:32.023686 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.023605 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0e5224e-5b53-4e41-9f1b-c942dc08b3e3" containerName="authorino" Jul 03 20:41:32.023686 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.023615 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0e5224e-5b53-4e41-9f1b-c942dc08b3e3" containerName="authorino" Jul 03 20:41:32.023686 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.023629 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3935174b-d229-4aff-b905-77f42068a7fa" containerName="authorino" Jul 03 20:41:32.023686 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.023635 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3935174b-d229-4aff-b905-77f42068a7fa" containerName="authorino" Jul 03 20:41:32.023843 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.023691 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3935174b-d229-4aff-b905-77f42068a7fa" containerName="authorino" Jul 03 20:41:32.023843 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.023698 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0e5224e-5b53-4e41-9f1b-c942dc08b3e3" containerName="authorino" Jul 03 20:41:32.028780 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.028733 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5776c5ff6-kl9l2" Jul 03 20:41:32.035961 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.035939 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 03 20:41:32.035961 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.035953 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zmmlw\"" Jul 03 20:41:32.036345 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.036327 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 03 20:41:32.044979 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.044948 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5776c5ff6-kl9l2"] Jul 03 20:41:32.120542 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.120508 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf49d3a7-a046-435b-8689-cf0a123b58af-webhook-cert\") pod \"maas-controller-5776c5ff6-kl9l2\" (UID: \"bf49d3a7-a046-435b-8689-cf0a123b58af\") " pod="opendatahub/maas-controller-5776c5ff6-kl9l2" Jul 03 20:41:32.120721 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.120550 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h88ng\" (UniqueName: \"kubernetes.io/projected/bf49d3a7-a046-435b-8689-cf0a123b58af-kube-api-access-h88ng\") pod \"maas-controller-5776c5ff6-kl9l2\" (UID: \"bf49d3a7-a046-435b-8689-cf0a123b58af\") " pod="opendatahub/maas-controller-5776c5ff6-kl9l2" Jul 03 20:41:32.218853 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.218817 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-746df7898c-wtksw"] Jul 03 20:41:32.221363 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.221337 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746df7898c-wtksw" Jul 03 20:41:32.221494 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.221455 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf49d3a7-a046-435b-8689-cf0a123b58af-webhook-cert\") pod \"maas-controller-5776c5ff6-kl9l2\" (UID: \"bf49d3a7-a046-435b-8689-cf0a123b58af\") " pod="opendatahub/maas-controller-5776c5ff6-kl9l2" Jul 03 20:41:32.221551 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.221492 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h88ng\" (UniqueName: \"kubernetes.io/projected/bf49d3a7-a046-435b-8689-cf0a123b58af-kube-api-access-h88ng\") pod \"maas-controller-5776c5ff6-kl9l2\" (UID: \"bf49d3a7-a046-435b-8689-cf0a123b58af\") " pod="opendatahub/maas-controller-5776c5ff6-kl9l2" Jul 03 20:41:32.224079 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.224057 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf49d3a7-a046-435b-8689-cf0a123b58af-webhook-cert\") pod \"maas-controller-5776c5ff6-kl9l2\" (UID: \"bf49d3a7-a046-435b-8689-cf0a123b58af\") " pod="opendatahub/maas-controller-5776c5ff6-kl9l2" Jul 03 20:41:32.229998 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.229977 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h88ng\" (UniqueName: \"kubernetes.io/projected/bf49d3a7-a046-435b-8689-cf0a123b58af-kube-api-access-h88ng\") pod \"maas-controller-5776c5ff6-kl9l2\" (UID: \"bf49d3a7-a046-435b-8689-cf0a123b58af\") " pod="opendatahub/maas-controller-5776c5ff6-kl9l2" Jul 03 20:41:32.232652 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.232631 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-746df7898c-wtksw"] Jul 03 20:41:32.322528 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.322450 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff905905-e0e1-453e-8cf8-5aeb443b0d08-webhook-cert\") pod \"maas-controller-746df7898c-wtksw\" (UID: \"ff905905-e0e1-453e-8cf8-5aeb443b0d08\") " pod="opendatahub/maas-controller-746df7898c-wtksw" Jul 03 20:41:32.322528 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.322526 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvfkc\" (UniqueName: \"kubernetes.io/projected/ff905905-e0e1-453e-8cf8-5aeb443b0d08-kube-api-access-vvfkc\") pod \"maas-controller-746df7898c-wtksw\" (UID: \"ff905905-e0e1-453e-8cf8-5aeb443b0d08\") " pod="opendatahub/maas-controller-746df7898c-wtksw" Jul 03 20:41:32.339347 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.339323 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5776c5ff6-kl9l2" Jul 03 20:41:32.423314 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.423276 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff905905-e0e1-453e-8cf8-5aeb443b0d08-webhook-cert\") pod \"maas-controller-746df7898c-wtksw\" (UID: \"ff905905-e0e1-453e-8cf8-5aeb443b0d08\") " pod="opendatahub/maas-controller-746df7898c-wtksw" Jul 03 20:41:32.423446 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.423389 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvfkc\" (UniqueName: \"kubernetes.io/projected/ff905905-e0e1-453e-8cf8-5aeb443b0d08-kube-api-access-vvfkc\") pod \"maas-controller-746df7898c-wtksw\" (UID: \"ff905905-e0e1-453e-8cf8-5aeb443b0d08\") " pod="opendatahub/maas-controller-746df7898c-wtksw" Jul 03 20:41:32.426889 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.426860 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff905905-e0e1-453e-8cf8-5aeb443b0d08-webhook-cert\") pod \"maas-controller-746df7898c-wtksw\" (UID: \"ff905905-e0e1-453e-8cf8-5aeb443b0d08\") " pod="opendatahub/maas-controller-746df7898c-wtksw" Jul 03 20:41:32.432475 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.432455 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvfkc\" (UniqueName: \"kubernetes.io/projected/ff905905-e0e1-453e-8cf8-5aeb443b0d08-kube-api-access-vvfkc\") pod \"maas-controller-746df7898c-wtksw\" (UID: \"ff905905-e0e1-453e-8cf8-5aeb443b0d08\") " pod="opendatahub/maas-controller-746df7898c-wtksw" Jul 03 20:41:32.472377 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.472349 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5776c5ff6-kl9l2"] Jul 03 20:41:32.475342 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:41:32.475321 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf49d3a7_a046_435b_8689_cf0a123b58af.slice/crio-b0a09bacdfa964d6cc1559f3b101eac7fd4519c2681f4ab485e180e18409b737 WatchSource:0}: Error finding container b0a09bacdfa964d6cc1559f3b101eac7fd4519c2681f4ab485e180e18409b737: Status 404 returned error can't find the container with id b0a09bacdfa964d6cc1559f3b101eac7fd4519c2681f4ab485e180e18409b737 Jul 03 20:41:32.546770 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.546720 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746df7898c-wtksw" Jul 03 20:41:32.678492 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:32.678321 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-746df7898c-wtksw"] Jul 03 20:41:32.680474 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:41:32.680453 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff905905_e0e1_453e_8cf8_5aeb443b0d08.slice/crio-c5e5b2c08f37ce3bb18dc55564466b3142460209de42d840ce1dad1a02441b78 WatchSource:0}: Error finding container c5e5b2c08f37ce3bb18dc55564466b3142460209de42d840ce1dad1a02441b78: Status 404 returned error can't find the container with id c5e5b2c08f37ce3bb18dc55564466b3142460209de42d840ce1dad1a02441b78 Jul 03 20:41:33.316010 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:33.315967 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5776c5ff6-kl9l2" event={"ID":"bf49d3a7-a046-435b-8689-cf0a123b58af","Type":"ContainerStarted","Data":"b0a09bacdfa964d6cc1559f3b101eac7fd4519c2681f4ab485e180e18409b737"} Jul 03 20:41:33.317936 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:33.317907 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746df7898c-wtksw" event={"ID":"ff905905-e0e1-453e-8cf8-5aeb443b0d08","Type":"ContainerStarted","Data":"c5e5b2c08f37ce3bb18dc55564466b3142460209de42d840ce1dad1a02441b78"} Jul 03 20:41:35.334928 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:35.334879 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746df7898c-wtksw" event={"ID":"ff905905-e0e1-453e-8cf8-5aeb443b0d08","Type":"ContainerStarted","Data":"3ae40ebc1ad94903464c31912f5e2c7aa048c31b978069909f1567fbda3f1eef"} Jul 03 20:41:35.344157 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:35.344123 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-746df7898c-wtksw" Jul 03 20:41:35.349155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:35.349117 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5776c5ff6-kl9l2" event={"ID":"bf49d3a7-a046-435b-8689-cf0a123b58af","Type":"ContainerStarted","Data":"6499d0811e8424fc858f2518f8adb840f7bfd2a67f75fcbb5fc99a9dd235e413"} Jul 03 20:41:35.349349 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:35.349330 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5776c5ff6-kl9l2" Jul 03 20:41:35.368236 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:35.368186 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-746df7898c-wtksw" podStartSLOduration=0.862723141 podStartE2EDuration="3.368171174s" totalImagesPullingTime="2.505448033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:41:35.366790639 +0000 UTC m=+561.825063075" watchObservedRunningTime="2026-07-03 20:41:35.368171174 +0000 UTC m=+561.826443645" Jul 03 20:41:35.393854 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:35.393801 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5776c5ff6-kl9l2" podStartSLOduration=0.679789081 podStartE2EDuration="3.393786041s" totalImagesPullingTime="2.71399696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:41:35.392306112 +0000 UTC m=+561.850578554" watchObservedRunningTime="2026-07-03 20:41:35.393786041 +0000 UTC m=+561.852058479" Jul 03 20:41:46.361158 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:46.361115 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-746df7898c-wtksw" Jul 03 20:41:46.361580 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:46.361556 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5776c5ff6-kl9l2" Jul 03 20:41:46.422068 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:46.422040 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5776c5ff6-kl9l2"] Jul 03 20:41:46.422259 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:46.422232 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5776c5ff6-kl9l2" podUID="bf49d3a7-a046-435b-8689-cf0a123b58af" containerName="manager" containerID="cri-o://6499d0811e8424fc858f2518f8adb840f7bfd2a67f75fcbb5fc99a9dd235e413" gracePeriod=30 Jul 03 20:41:46.673229 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:46.673204 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5776c5ff6-kl9l2" Jul 03 20:41:46.739976 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:46.739943 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf49d3a7-a046-435b-8689-cf0a123b58af-webhook-cert\") pod \"bf49d3a7-a046-435b-8689-cf0a123b58af\" (UID: \"bf49d3a7-a046-435b-8689-cf0a123b58af\") " Jul 03 20:41:46.740143 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:46.740011 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h88ng\" (UniqueName: \"kubernetes.io/projected/bf49d3a7-a046-435b-8689-cf0a123b58af-kube-api-access-h88ng\") pod \"bf49d3a7-a046-435b-8689-cf0a123b58af\" (UID: \"bf49d3a7-a046-435b-8689-cf0a123b58af\") " Jul 03 20:41:46.742268 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:46.742242 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf49d3a7-a046-435b-8689-cf0a123b58af-kube-api-access-h88ng" (OuterVolumeSpecName: "kube-api-access-h88ng") pod "bf49d3a7-a046-435b-8689-cf0a123b58af" (UID: "bf49d3a7-a046-435b-8689-cf0a123b58af"). InnerVolumeSpecName "kube-api-access-h88ng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:41:46.742268 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:46.742259 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf49d3a7-a046-435b-8689-cf0a123b58af-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "bf49d3a7-a046-435b-8689-cf0a123b58af" (UID: "bf49d3a7-a046-435b-8689-cf0a123b58af"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:41:46.841422 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:46.841398 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h88ng\" (UniqueName: \"kubernetes.io/projected/bf49d3a7-a046-435b-8689-cf0a123b58af-kube-api-access-h88ng\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:41:46.841422 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:46.841421 2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf49d3a7-a046-435b-8689-cf0a123b58af-webhook-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:41:47.403876 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:47.403844 2582 generic.go:356] "Generic (PLEG): container finished" podID="bf49d3a7-a046-435b-8689-cf0a123b58af" containerID="6499d0811e8424fc858f2518f8adb840f7bfd2a67f75fcbb5fc99a9dd235e413" exitCode=0 Jul 03 20:41:47.404292 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:47.403887 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5776c5ff6-kl9l2" Jul 03 20:41:47.404292 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:47.403903 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5776c5ff6-kl9l2" event={"ID":"bf49d3a7-a046-435b-8689-cf0a123b58af","Type":"ContainerDied","Data":"6499d0811e8424fc858f2518f8adb840f7bfd2a67f75fcbb5fc99a9dd235e413"} Jul 03 20:41:47.404292 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:47.403943 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5776c5ff6-kl9l2" event={"ID":"bf49d3a7-a046-435b-8689-cf0a123b58af","Type":"ContainerDied","Data":"b0a09bacdfa964d6cc1559f3b101eac7fd4519c2681f4ab485e180e18409b737"} Jul 03 20:41:47.404292 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:47.403967 2582 scope.go:117] "RemoveContainer" containerID="6499d0811e8424fc858f2518f8adb840f7bfd2a67f75fcbb5fc99a9dd235e413" Jul 03 20:41:47.412573 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:47.412558 2582 scope.go:117] "RemoveContainer" containerID="6499d0811e8424fc858f2518f8adb840f7bfd2a67f75fcbb5fc99a9dd235e413" Jul 03 20:41:47.412810 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:41:47.412794 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6499d0811e8424fc858f2518f8adb840f7bfd2a67f75fcbb5fc99a9dd235e413\": container with ID starting with 6499d0811e8424fc858f2518f8adb840f7bfd2a67f75fcbb5fc99a9dd235e413 not found: ID does not exist" containerID="6499d0811e8424fc858f2518f8adb840f7bfd2a67f75fcbb5fc99a9dd235e413" Jul 03 20:41:47.412849 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:47.412818 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6499d0811e8424fc858f2518f8adb840f7bfd2a67f75fcbb5fc99a9dd235e413"} err="failed to get container status \"6499d0811e8424fc858f2518f8adb840f7bfd2a67f75fcbb5fc99a9dd235e413\": rpc error: code = NotFound desc = could not find container \"6499d0811e8424fc858f2518f8adb840f7bfd2a67f75fcbb5fc99a9dd235e413\": container with ID starting with 6499d0811e8424fc858f2518f8adb840f7bfd2a67f75fcbb5fc99a9dd235e413 not found: ID does not exist" Jul 03 20:41:47.425373 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:47.425348 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5776c5ff6-kl9l2"] Jul 03 20:41:47.432770 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:47.431380 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5776c5ff6-kl9l2"] Jul 03 20:41:47.986533 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:41:47.986501 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf49d3a7-a046-435b-8689-cf0a123b58af" path="/var/lib/kubelet/pods/bf49d3a7-a046-435b-8689-cf0a123b58af/volumes" Jul 03 20:42:07.236811 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.236777 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-594b7cb88d-s82mn"] Jul 03 20:42:07.237269 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.237088 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf49d3a7-a046-435b-8689-cf0a123b58af" containerName="manager" Jul 03 20:42:07.237269 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.237098 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf49d3a7-a046-435b-8689-cf0a123b58af" containerName="manager" Jul 03 20:42:07.237269 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.237173 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf49d3a7-a046-435b-8689-cf0a123b58af" containerName="manager" Jul 03 20:42:07.240038 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.240017 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-594b7cb88d-s82mn" Jul 03 20:42:07.242680 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.242650 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-pjhz5\"" Jul 03 20:42:07.243593 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.243572 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 03 20:42:07.253015 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.252997 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-594b7cb88d-s82mn"] Jul 03 20:42:07.311976 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.311947 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e98bdf4-940b-40f3-97ab-b3d113e96590-maas-api-tls\") pod \"maas-api-594b7cb88d-s82mn\" (UID: \"5e98bdf4-940b-40f3-97ab-b3d113e96590\") " pod="opendatahub/maas-api-594b7cb88d-s82mn" Jul 03 20:42:07.312163 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.311996 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbb6h\" (UniqueName: \"kubernetes.io/projected/5e98bdf4-940b-40f3-97ab-b3d113e96590-kube-api-access-kbb6h\") pod \"maas-api-594b7cb88d-s82mn\" (UID: \"5e98bdf4-940b-40f3-97ab-b3d113e96590\") " pod="opendatahub/maas-api-594b7cb88d-s82mn" Jul 03 20:42:07.312163 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.312021 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e98bdf4-940b-40f3-97ab-b3d113e96590-openshift-service-ca\") pod \"maas-api-594b7cb88d-s82mn\" (UID: \"5e98bdf4-940b-40f3-97ab-b3d113e96590\") " pod="opendatahub/maas-api-594b7cb88d-s82mn" Jul 03 20:42:07.413177 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.413138 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e98bdf4-940b-40f3-97ab-b3d113e96590-maas-api-tls\") pod \"maas-api-594b7cb88d-s82mn\" (UID: \"5e98bdf4-940b-40f3-97ab-b3d113e96590\") " pod="opendatahub/maas-api-594b7cb88d-s82mn" Jul 03 20:42:07.413365 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.413198 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbb6h\" (UniqueName: \"kubernetes.io/projected/5e98bdf4-940b-40f3-97ab-b3d113e96590-kube-api-access-kbb6h\") pod \"maas-api-594b7cb88d-s82mn\" (UID: \"5e98bdf4-940b-40f3-97ab-b3d113e96590\") " pod="opendatahub/maas-api-594b7cb88d-s82mn" Jul 03 20:42:07.413365 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.413229 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e98bdf4-940b-40f3-97ab-b3d113e96590-openshift-service-ca\") pod \"maas-api-594b7cb88d-s82mn\" (UID: \"5e98bdf4-940b-40f3-97ab-b3d113e96590\") " pod="opendatahub/maas-api-594b7cb88d-s82mn" Jul 03 20:42:07.413907 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.413882 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e98bdf4-940b-40f3-97ab-b3d113e96590-openshift-service-ca\") pod \"maas-api-594b7cb88d-s82mn\" (UID: \"5e98bdf4-940b-40f3-97ab-b3d113e96590\") " pod="opendatahub/maas-api-594b7cb88d-s82mn" Jul 03 20:42:07.415616 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.415596 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e98bdf4-940b-40f3-97ab-b3d113e96590-maas-api-tls\") pod \"maas-api-594b7cb88d-s82mn\" (UID: \"5e98bdf4-940b-40f3-97ab-b3d113e96590\") " pod="opendatahub/maas-api-594b7cb88d-s82mn" Jul 03 20:42:07.424406 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.424384 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbb6h\" (UniqueName: \"kubernetes.io/projected/5e98bdf4-940b-40f3-97ab-b3d113e96590-kube-api-access-kbb6h\") pod \"maas-api-594b7cb88d-s82mn\" (UID: \"5e98bdf4-940b-40f3-97ab-b3d113e96590\") " pod="opendatahub/maas-api-594b7cb88d-s82mn" Jul 03 20:42:07.549877 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.549789 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-594b7cb88d-s82mn" Jul 03 20:42:07.703550 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.703521 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-594b7cb88d-s82mn"] Jul 03 20:42:07.706222 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:42:07.706186 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e98bdf4_940b_40f3_97ab_b3d113e96590.slice/crio-9c6b6648821fa4b9bc03f111b20697dd31811781d067c753f917ddc1952c40cb WatchSource:0}: Error finding container 9c6b6648821fa4b9bc03f111b20697dd31811781d067c753f917ddc1952c40cb: Status 404 returned error can't find the container with id 9c6b6648821fa4b9bc03f111b20697dd31811781d067c753f917ddc1952c40cb Jul 03 20:42:07.812779 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.812675 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-p4gh8"] Jul 03 20:42:07.816931 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.816911 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-p4gh8" Jul 03 20:42:07.822731 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.822702 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-g2mtn\"" Jul 03 20:42:07.822870 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.822764 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 03 20:42:07.832417 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.832392 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-p4gh8"] Jul 03 20:42:07.917246 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.917205 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc73801b-35f6-45d9-992c-ca9b9ca8fc00-plugins-config-volume\") pod \"payload-processing-58f86c7798-p4gh8\" (UID: \"cc73801b-35f6-45d9-992c-ca9b9ca8fc00\") " pod="openshift-ingress/payload-processing-58f86c7798-p4gh8" Jul 03 20:42:07.917422 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:07.917259 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjmnw\" (UniqueName: \"kubernetes.io/projected/cc73801b-35f6-45d9-992c-ca9b9ca8fc00-kube-api-access-kjmnw\") pod \"payload-processing-58f86c7798-p4gh8\" (UID: \"cc73801b-35f6-45d9-992c-ca9b9ca8fc00\") " pod="openshift-ingress/payload-processing-58f86c7798-p4gh8" Jul 03 20:42:08.018212 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:08.018175 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc73801b-35f6-45d9-992c-ca9b9ca8fc00-plugins-config-volume\") pod \"payload-processing-58f86c7798-p4gh8\" (UID: \"cc73801b-35f6-45d9-992c-ca9b9ca8fc00\") " pod="openshift-ingress/payload-processing-58f86c7798-p4gh8" Jul 03 20:42:08.018386 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:08.018225 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjmnw\" (UniqueName: \"kubernetes.io/projected/cc73801b-35f6-45d9-992c-ca9b9ca8fc00-kube-api-access-kjmnw\") pod \"payload-processing-58f86c7798-p4gh8\" (UID: \"cc73801b-35f6-45d9-992c-ca9b9ca8fc00\") " pod="openshift-ingress/payload-processing-58f86c7798-p4gh8" Jul 03 20:42:08.019001 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:08.018982 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc73801b-35f6-45d9-992c-ca9b9ca8fc00-plugins-config-volume\") pod \"payload-processing-58f86c7798-p4gh8\" (UID: \"cc73801b-35f6-45d9-992c-ca9b9ca8fc00\") " pod="openshift-ingress/payload-processing-58f86c7798-p4gh8" Jul 03 20:42:08.026502 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:08.026476 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjmnw\" (UniqueName: \"kubernetes.io/projected/cc73801b-35f6-45d9-992c-ca9b9ca8fc00-kube-api-access-kjmnw\") pod \"payload-processing-58f86c7798-p4gh8\" (UID: \"cc73801b-35f6-45d9-992c-ca9b9ca8fc00\") " pod="openshift-ingress/payload-processing-58f86c7798-p4gh8" Jul 03 20:42:08.128132 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:08.128029 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-p4gh8" Jul 03 20:42:08.263705 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:08.263672 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-p4gh8"] Jul 03 20:42:08.267043 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:42:08.267015 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc73801b_35f6_45d9_992c_ca9b9ca8fc00.slice/crio-38b3160382efc56dd8aaff4563dc74f7130732ee010bcecc2774c2ddddd01389 WatchSource:0}: Error finding container 38b3160382efc56dd8aaff4563dc74f7130732ee010bcecc2774c2ddddd01389: Status 404 returned error can't find the container with id 38b3160382efc56dd8aaff4563dc74f7130732ee010bcecc2774c2ddddd01389 Jul 03 20:42:08.499765 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:08.499717 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-p4gh8" event={"ID":"cc73801b-35f6-45d9-992c-ca9b9ca8fc00","Type":"ContainerStarted","Data":"38b3160382efc56dd8aaff4563dc74f7130732ee010bcecc2774c2ddddd01389"} Jul 03 20:42:08.501058 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:08.501034 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-594b7cb88d-s82mn" event={"ID":"5e98bdf4-940b-40f3-97ab-b3d113e96590","Type":"ContainerStarted","Data":"9c6b6648821fa4b9bc03f111b20697dd31811781d067c753f917ddc1952c40cb"} Jul 03 20:42:09.509737 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:09.509700 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-594b7cb88d-s82mn" event={"ID":"5e98bdf4-940b-40f3-97ab-b3d113e96590","Type":"ContainerStarted","Data":"351deb5eed4fb8ba3352cc0815ffde9b06537012df3f235122776e4e0210f3e6"} Jul 03 20:42:09.510149 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:09.509818 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-594b7cb88d-s82mn" Jul 03 20:42:09.526486 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:09.526438 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-594b7cb88d-s82mn" podStartSLOduration=1.133061121 podStartE2EDuration="2.526421179s" totalImagesPullingTime="1.393360058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:42:09.525794668 +0000 UTC m=+595.984067108" watchObservedRunningTime="2026-07-03 20:42:09.526421179 +0000 UTC m=+595.984693621" Jul 03 20:42:10.515682 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:10.515643 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-p4gh8" event={"ID":"cc73801b-35f6-45d9-992c-ca9b9ca8fc00","Type":"ContainerStarted","Data":"aaa8264c03bbeefb3ea61460fd7d49df3be30e08a098abdc63cc85eeadc09eaf"} Jul 03 20:42:10.516150 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:10.515951 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-p4gh8" Jul 03 20:42:10.531763 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:10.531697 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-p4gh8" podStartSLOduration=2.233639455 podStartE2EDuration="3.531681253s" totalImagesPullingTime="1.298041798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:42:10.530540342 +0000 UTC m=+596.988812782" watchObservedRunningTime="2026-07-03 20:42:10.531681253 +0000 UTC m=+596.989953692" Jul 03 20:42:15.521034 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:15.521006 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-594b7cb88d-s82mn" Jul 03 20:42:21.522936 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:21.522905 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-p4gh8" Jul 03 20:42:55.840857 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:55.840823 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj"] Jul 03 20:42:55.846563 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:55.845958 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:55.851102 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:55.851081 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4vtgc\"" Jul 03 20:42:55.851736 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:55.851132 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 03 20:42:55.851736 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:55.851251 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 03 20:42:55.851736 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:55.851149 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 03 20:42:55.854695 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:55.854675 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj"] Jul 03 20:42:55.910567 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:55.910532 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsb64\" (UniqueName: \"kubernetes.io/projected/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-kube-api-access-lsb64\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:55.910567 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:55.910569 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:55.910794 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:55.910634 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:55.910794 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:55.910665 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:55.910794 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:55.910683 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:55.910794 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:55.910773 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:56.011993 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.011967 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:56.012179 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.012014 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:56.012179 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.012038 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:56.012179 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.012072 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:56.012179 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.012119 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsb64\" (UniqueName: \"kubernetes.io/projected/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-kube-api-access-lsb64\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:56.012179 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.012146 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:56.012460 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.012423 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:56.012517 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.012488 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:56.012565 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.012504 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:56.014495 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.014466 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:56.014803 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.014785 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:56.019650 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.019613 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsb64\" (UniqueName: \"kubernetes.io/projected/bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23-kube-api-access-lsb64\") pod \"e2e-distinct-simulated-kserve-59975d69f9-sq6pj\" (UID: \"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:56.160541 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.160503 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:42:56.296224 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.296193 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj"] Jul 03 20:42:56.299479 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:42:56.299449 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf8875b8_47b9_4c8e_a76f_1f9c9bc08f23.slice/crio-be7535c05c4dbb1ca0e5d048eb3e4b53a91317a4e2b7acfa59c4c7c9e34fc21a WatchSource:0}: Error finding container be7535c05c4dbb1ca0e5d048eb3e4b53a91317a4e2b7acfa59c4c7c9e34fc21a: Status 404 returned error can't find the container with id be7535c05c4dbb1ca0e5d048eb3e4b53a91317a4e2b7acfa59c4c7c9e34fc21a Jul 03 20:42:56.300731 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.300715 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 20:42:56.730741 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:56.730702 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" event={"ID":"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23","Type":"ContainerStarted","Data":"be7535c05c4dbb1ca0e5d048eb3e4b53a91317a4e2b7acfa59c4c7c9e34fc21a"} Jul 03 20:42:57.736537 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:57.736461 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" event={"ID":"bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23","Type":"ContainerStarted","Data":"8d60177bea5d5da422e53240a100d3ea014e353c380df3d430e2ddf5f68355cc"} Jul 03 20:42:57.754422 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:42:57.754380 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" podStartSLOduration=1.648183098 podStartE2EDuration="2.754364778s" totalImagesPullingTime="1.10618168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:42:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:42:57.753459665 +0000 UTC m=+644.211732098" watchObservedRunningTime="2026-07-03 20:42:57.754364778 +0000 UTC m=+644.212637216" Jul 03 20:43:06.160936 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:06.160905 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:43:06.161329 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:06.160954 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:43:06.173174 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:06.173150 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:43:06.790078 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:06.790052 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-sq6pj" Jul 03 20:43:49.855896 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:49.855860 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r"] Jul 03 20:43:49.859530 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:49.859504 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:49.863316 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:49.863294 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 03 20:43:49.875723 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:49.875701 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r"] Jul 03 20:43:49.982455 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:49.982421 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:49.982633 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:49.982462 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:49.982633 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:49.982490 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:49.982633 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:49.982532 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:49.982633 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:49.982555 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:49.982633 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:49.982621 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72lgg\" (UniqueName: \"kubernetes.io/projected/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-kube-api-access-72lgg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:50.083760 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.083703 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:50.083760 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.083761 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:50.084004 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.083858 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72lgg\" (UniqueName: \"kubernetes.io/projected/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-kube-api-access-72lgg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:50.084004 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.083942 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:50.084004 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.083974 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:50.084004 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.083998 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:50.084298 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.084264 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:50.084429 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.084406 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:50.084586 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.084560 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:50.086579 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.086560 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:50.086579 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.086574 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:50.093194 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.093175 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72lgg\" (UniqueName: \"kubernetes.io/projected/e34f6aa7-dcbd-4a0e-9246-6405bcadee0e-kube-api-access-72lgg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mpg6r\" (UID: \"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:50.169132 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.169097 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:43:50.304647 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.304623 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r"] Jul 03 20:43:50.307400 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:43:50.307358 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode34f6aa7_dcbd_4a0e_9246_6405bcadee0e.slice/crio-576af839795c651b4827614873d0e06b037ad4bc9d5b182362114e7cdbad96b4 WatchSource:0}: Error finding container 576af839795c651b4827614873d0e06b037ad4bc9d5b182362114e7cdbad96b4: Status 404 returned error can't find the container with id 576af839795c651b4827614873d0e06b037ad4bc9d5b182362114e7cdbad96b4 Jul 03 20:43:50.979063 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.979027 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" event={"ID":"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e","Type":"ContainerStarted","Data":"7079a0c39e6e299e9c5236467251b53f7f479ee8bfc24c66120ae3ade8c52e27"} Jul 03 20:43:50.979439 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.979091 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" event={"ID":"e34f6aa7-dcbd-4a0e-9246-6405bcadee0e","Type":"ContainerStarted","Data":"576af839795c651b4827614873d0e06b037ad4bc9d5b182362114e7cdbad96b4"} Jul 03 20:43:50.997142 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:43:50.997083 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" podStartSLOduration=1.8338687870000001 podStartE2EDuration="1.997067889s" totalImagesPullingTime="163.199102ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:43:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:43:50.995384771 +0000 UTC m=+697.453657210" watchObservedRunningTime="2026-07-03 20:43:50.997067889 +0000 UTC m=+697.455340327" Jul 03 20:44:00.169372 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:44:00.169336 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:44:00.169853 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:44:00.169403 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:44:00.182096 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:44:00.182075 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:44:01.037055 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:44:01.037028 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mpg6r" Jul 03 20:45:49.689256 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:49.689137 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65bcd68d89-jbsh9"] Jul 03 20:45:49.689845 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:49.689392 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-65bcd68d89-jbsh9" podUID="51811eed-c619-430a-8fc7-28dc31177876" containerName="authorino" containerID="cri-o://7a43ed4ac020792437592dfe6b30a9de3fbd73bb38514d626998fc005c48bd89" gracePeriod=30 Jul 03 20:45:49.941614 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:49.941556 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65bcd68d89-jbsh9" Jul 03 20:45:50.073856 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.073819 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/51811eed-c619-430a-8fc7-28dc31177876-tls-cert\") pod \"51811eed-c619-430a-8fc7-28dc31177876\" (UID: \"51811eed-c619-430a-8fc7-28dc31177876\") " Jul 03 20:45:50.074041 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.073890 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7hwh\" (UniqueName: \"kubernetes.io/projected/51811eed-c619-430a-8fc7-28dc31177876-kube-api-access-d7hwh\") pod \"51811eed-c619-430a-8fc7-28dc31177876\" (UID: \"51811eed-c619-430a-8fc7-28dc31177876\") " Jul 03 20:45:50.076103 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.076063 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51811eed-c619-430a-8fc7-28dc31177876-kube-api-access-d7hwh" (OuterVolumeSpecName: "kube-api-access-d7hwh") pod "51811eed-c619-430a-8fc7-28dc31177876" (UID: "51811eed-c619-430a-8fc7-28dc31177876"). InnerVolumeSpecName "kube-api-access-d7hwh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:45:50.084681 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.084652 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51811eed-c619-430a-8fc7-28dc31177876-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "51811eed-c619-430a-8fc7-28dc31177876" (UID: "51811eed-c619-430a-8fc7-28dc31177876"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:45:50.175490 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.175455 2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/51811eed-c619-430a-8fc7-28dc31177876-tls-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:45:50.175490 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.175485 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7hwh\" (UniqueName: \"kubernetes.io/projected/51811eed-c619-430a-8fc7-28dc31177876-kube-api-access-d7hwh\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:45:50.515562 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.515524 2582 generic.go:356] "Generic (PLEG): container finished" podID="51811eed-c619-430a-8fc7-28dc31177876" containerID="7a43ed4ac020792437592dfe6b30a9de3fbd73bb38514d626998fc005c48bd89" exitCode=0 Jul 03 20:45:50.515793 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.515585 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65bcd68d89-jbsh9" Jul 03 20:45:50.515793 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.515613 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65bcd68d89-jbsh9" event={"ID":"51811eed-c619-430a-8fc7-28dc31177876","Type":"ContainerDied","Data":"7a43ed4ac020792437592dfe6b30a9de3fbd73bb38514d626998fc005c48bd89"} Jul 03 20:45:50.515793 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.515659 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65bcd68d89-jbsh9" event={"ID":"51811eed-c619-430a-8fc7-28dc31177876","Type":"ContainerDied","Data":"c5e3b698dcd445f60013e2140944998e07955bc2c07e13b776136fc0729d41e9"} Jul 03 20:45:50.515793 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.515676 2582 scope.go:117] "RemoveContainer" containerID="7a43ed4ac020792437592dfe6b30a9de3fbd73bb38514d626998fc005c48bd89" Jul 03 20:45:50.525574 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.525554 2582 scope.go:117] "RemoveContainer" containerID="7a43ed4ac020792437592dfe6b30a9de3fbd73bb38514d626998fc005c48bd89" Jul 03 20:45:50.525908 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:45:50.525889 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a43ed4ac020792437592dfe6b30a9de3fbd73bb38514d626998fc005c48bd89\": container with ID starting with 7a43ed4ac020792437592dfe6b30a9de3fbd73bb38514d626998fc005c48bd89 not found: ID does not exist" containerID="7a43ed4ac020792437592dfe6b30a9de3fbd73bb38514d626998fc005c48bd89" Jul 03 20:45:50.525988 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.525920 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a43ed4ac020792437592dfe6b30a9de3fbd73bb38514d626998fc005c48bd89"} err="failed to get container status \"7a43ed4ac020792437592dfe6b30a9de3fbd73bb38514d626998fc005c48bd89\": rpc error: code = NotFound desc = could not find container \"7a43ed4ac020792437592dfe6b30a9de3fbd73bb38514d626998fc005c48bd89\": container with ID starting with 7a43ed4ac020792437592dfe6b30a9de3fbd73bb38514d626998fc005c48bd89 not found: ID does not exist" Jul 03 20:45:50.543722 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.543690 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65bcd68d89-jbsh9"] Jul 03 20:45:50.549831 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:50.549806 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-65bcd68d89-jbsh9"] Jul 03 20:45:51.987698 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:45:51.987665 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51811eed-c619-430a-8fc7-28dc31177876" path="/var/lib/kubelet/pods/51811eed-c619-430a-8fc7-28dc31177876/volumes" Jul 03 20:49:02.558769 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:02.558666 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-746df7898c-wtksw"] Jul 03 20:49:02.561193 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:02.558937 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-746df7898c-wtksw" podUID="ff905905-e0e1-453e-8cf8-5aeb443b0d08" containerName="manager" containerID="cri-o://3ae40ebc1ad94903464c31912f5e2c7aa048c31b978069909f1567fbda3f1eef" gracePeriod=30 Jul 03 20:49:02.837449 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:02.837426 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746df7898c-wtksw" Jul 03 20:49:02.879805 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:02.879773 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff905905-e0e1-453e-8cf8-5aeb443b0d08-webhook-cert\") pod \"ff905905-e0e1-453e-8cf8-5aeb443b0d08\" (UID: \"ff905905-e0e1-453e-8cf8-5aeb443b0d08\") " Jul 03 20:49:02.879987 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:02.879838 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvfkc\" (UniqueName: \"kubernetes.io/projected/ff905905-e0e1-453e-8cf8-5aeb443b0d08-kube-api-access-vvfkc\") pod \"ff905905-e0e1-453e-8cf8-5aeb443b0d08\" (UID: \"ff905905-e0e1-453e-8cf8-5aeb443b0d08\") " Jul 03 20:49:02.882000 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:02.881974 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff905905-e0e1-453e-8cf8-5aeb443b0d08-kube-api-access-vvfkc" (OuterVolumeSpecName: "kube-api-access-vvfkc") pod "ff905905-e0e1-453e-8cf8-5aeb443b0d08" (UID: "ff905905-e0e1-453e-8cf8-5aeb443b0d08"). InnerVolumeSpecName "kube-api-access-vvfkc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:49:02.882100 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:02.882056 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff905905-e0e1-453e-8cf8-5aeb443b0d08-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ff905905-e0e1-453e-8cf8-5aeb443b0d08" (UID: "ff905905-e0e1-453e-8cf8-5aeb443b0d08"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:49:02.981046 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:02.981001 2582 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff905905-e0e1-453e-8cf8-5aeb443b0d08-webhook-cert\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:49:02.981046 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:02.981042 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvfkc\" (UniqueName: \"kubernetes.io/projected/ff905905-e0e1-453e-8cf8-5aeb443b0d08-kube-api-access-vvfkc\") on node \"ip-10-0-139-68.ec2.internal\" DevicePath \"\"" Jul 03 20:49:03.399629 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:03.399593 2582 generic.go:356] "Generic (PLEG): container finished" podID="ff905905-e0e1-453e-8cf8-5aeb443b0d08" containerID="3ae40ebc1ad94903464c31912f5e2c7aa048c31b978069909f1567fbda3f1eef" exitCode=0 Jul 03 20:49:03.399860 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:03.399637 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746df7898c-wtksw" Jul 03 20:49:03.399860 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:03.399660 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746df7898c-wtksw" event={"ID":"ff905905-e0e1-453e-8cf8-5aeb443b0d08","Type":"ContainerDied","Data":"3ae40ebc1ad94903464c31912f5e2c7aa048c31b978069909f1567fbda3f1eef"} Jul 03 20:49:03.399860 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:03.399712 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746df7898c-wtksw" event={"ID":"ff905905-e0e1-453e-8cf8-5aeb443b0d08","Type":"ContainerDied","Data":"c5e5b2c08f37ce3bb18dc55564466b3142460209de42d840ce1dad1a02441b78"} Jul 03 20:49:03.399860 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:03.399740 2582 scope.go:117] "RemoveContainer" containerID="3ae40ebc1ad94903464c31912f5e2c7aa048c31b978069909f1567fbda3f1eef" Jul 03 20:49:03.408512 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:03.408489 2582 scope.go:117] "RemoveContainer" containerID="3ae40ebc1ad94903464c31912f5e2c7aa048c31b978069909f1567fbda3f1eef" Jul 03 20:49:03.408772 ip-10-0-139-68 kubenswrapper[2582]: E0703 20:49:03.408737 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ae40ebc1ad94903464c31912f5e2c7aa048c31b978069909f1567fbda3f1eef\": container with ID starting with 3ae40ebc1ad94903464c31912f5e2c7aa048c31b978069909f1567fbda3f1eef not found: ID does not exist" containerID="3ae40ebc1ad94903464c31912f5e2c7aa048c31b978069909f1567fbda3f1eef" Jul 03 20:49:03.408841 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:03.408780 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ae40ebc1ad94903464c31912f5e2c7aa048c31b978069909f1567fbda3f1eef"} err="failed to get container status \"3ae40ebc1ad94903464c31912f5e2c7aa048c31b978069909f1567fbda3f1eef\": rpc error: code = NotFound desc = could not find container \"3ae40ebc1ad94903464c31912f5e2c7aa048c31b978069909f1567fbda3f1eef\": container with ID starting with 3ae40ebc1ad94903464c31912f5e2c7aa048c31b978069909f1567fbda3f1eef not found: ID does not exist" Jul 03 20:49:03.420679 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:03.420655 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-746df7898c-wtksw"] Jul 03 20:49:03.424892 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:03.424872 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-746df7898c-wtksw"] Jul 03 20:49:03.985794 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:03.985740 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff905905-e0e1-453e-8cf8-5aeb443b0d08" path="/var/lib/kubelet/pods/ff905905-e0e1-453e-8cf8-5aeb443b0d08/volumes" Jul 03 20:49:04.200326 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.200296 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-746df7898c-ts9ms"] Jul 03 20:49:04.200641 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.200629 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff905905-e0e1-453e-8cf8-5aeb443b0d08" containerName="manager" Jul 03 20:49:04.200686 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.200644 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff905905-e0e1-453e-8cf8-5aeb443b0d08" containerName="manager" Jul 03 20:49:04.200686 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.200652 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51811eed-c619-430a-8fc7-28dc31177876" containerName="authorino" Jul 03 20:49:04.200686 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.200657 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="51811eed-c619-430a-8fc7-28dc31177876" containerName="authorino" Jul 03 20:49:04.200803 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.200712 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff905905-e0e1-453e-8cf8-5aeb443b0d08" containerName="manager" Jul 03 20:49:04.200803 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.200720 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="51811eed-c619-430a-8fc7-28dc31177876" containerName="authorino" Jul 03 20:49:04.205077 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.205057 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746df7898c-ts9ms" Jul 03 20:49:04.208331 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.208306 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zmmlw\"" Jul 03 20:49:04.208331 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.208320 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 03 20:49:04.208520 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.208370 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 03 20:49:04.215282 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.215263 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-746df7898c-ts9ms"] Jul 03 20:49:04.290660 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.290585 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85950ae0-9eaf-435e-b19d-14c4c6172384-webhook-cert\") pod \"maas-controller-746df7898c-ts9ms\" (UID: \"85950ae0-9eaf-435e-b19d-14c4c6172384\") " pod="opendatahub/maas-controller-746df7898c-ts9ms" Jul 03 20:49:04.290660 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.290632 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbw9m\" (UniqueName: \"kubernetes.io/projected/85950ae0-9eaf-435e-b19d-14c4c6172384-kube-api-access-lbw9m\") pod \"maas-controller-746df7898c-ts9ms\" (UID: \"85950ae0-9eaf-435e-b19d-14c4c6172384\") " pod="opendatahub/maas-controller-746df7898c-ts9ms" Jul 03 20:49:04.391068 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.391035 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85950ae0-9eaf-435e-b19d-14c4c6172384-webhook-cert\") pod \"maas-controller-746df7898c-ts9ms\" (UID: \"85950ae0-9eaf-435e-b19d-14c4c6172384\") " pod="opendatahub/maas-controller-746df7898c-ts9ms" Jul 03 20:49:04.391233 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.391102 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbw9m\" (UniqueName: \"kubernetes.io/projected/85950ae0-9eaf-435e-b19d-14c4c6172384-kube-api-access-lbw9m\") pod \"maas-controller-746df7898c-ts9ms\" (UID: \"85950ae0-9eaf-435e-b19d-14c4c6172384\") " pod="opendatahub/maas-controller-746df7898c-ts9ms" Jul 03 20:49:04.393662 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.393634 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85950ae0-9eaf-435e-b19d-14c4c6172384-webhook-cert\") pod \"maas-controller-746df7898c-ts9ms\" (UID: \"85950ae0-9eaf-435e-b19d-14c4c6172384\") " pod="opendatahub/maas-controller-746df7898c-ts9ms" Jul 03 20:49:04.399204 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.399183 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbw9m\" (UniqueName: \"kubernetes.io/projected/85950ae0-9eaf-435e-b19d-14c4c6172384-kube-api-access-lbw9m\") pod \"maas-controller-746df7898c-ts9ms\" (UID: \"85950ae0-9eaf-435e-b19d-14c4c6172384\") " pod="opendatahub/maas-controller-746df7898c-ts9ms" Jul 03 20:49:04.515601 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.515577 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-746df7898c-ts9ms" Jul 03 20:49:04.855240 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.855212 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-746df7898c-ts9ms"] Jul 03 20:49:04.857235 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:49:04.857198 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85950ae0_9eaf_435e_b19d_14c4c6172384.slice/crio-7689d43a5f051ea4bca3ceeb7e9841328a8e1a647cf9f9132c6299da7c055ff5 WatchSource:0}: Error finding container 7689d43a5f051ea4bca3ceeb7e9841328a8e1a647cf9f9132c6299da7c055ff5: Status 404 returned error can't find the container with id 7689d43a5f051ea4bca3ceeb7e9841328a8e1a647cf9f9132c6299da7c055ff5 Jul 03 20:49:04.858505 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:04.858488 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 20:49:05.410295 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:05.410262 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746df7898c-ts9ms" event={"ID":"85950ae0-9eaf-435e-b19d-14c4c6172384","Type":"ContainerStarted","Data":"7689d43a5f051ea4bca3ceeb7e9841328a8e1a647cf9f9132c6299da7c055ff5"} Jul 03 20:49:06.416783 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:06.416734 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-746df7898c-ts9ms" event={"ID":"85950ae0-9eaf-435e-b19d-14c4c6172384","Type":"ContainerStarted","Data":"9c173edcdc51ea78aee7384c8c4175a4cbbb089a8ac9b1da3b859a2e55b56d03"} Jul 03 20:49:06.417212 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:06.416876 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-746df7898c-ts9ms" Jul 03 20:49:06.434508 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:06.434457 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-746df7898c-ts9ms" podStartSLOduration=1.919570805 podStartE2EDuration="2.434443494s" totalImagesPullingTime="514.872689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:49:06.432495613 +0000 UTC m=+1012.890768053" watchObservedRunningTime="2026-07-03 20:49:06.434443494 +0000 UTC m=+1012.892715933" Jul 03 20:49:17.427257 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:49:17.427225 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-746df7898c-ts9ms" Jul 03 20:57:37.939088 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:37.939016 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tvcsp_b1a66c17-ccf0-4802-86b0-b16795c603a4/manager/0.log" Jul 03 20:57:38.058391 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:38.058360 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-k27gj_bb5c5346-91f7-4f20-b3bf-e2beb1aec43b/manager/0.log" Jul 03 20:57:38.175062 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:38.175033 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-594b7cb88d-s82mn_5e98bdf4-940b-40f3-97ab-b3d113e96590/maas-api/0.log" Jul 03 20:57:38.415321 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:38.415295 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-746df7898c-ts9ms_85950ae0-9eaf-435e-b19d-14c4c6172384/manager/0.log" Jul 03 20:57:38.544833 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:38.544806 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-wd9c5_e0ed9b90-a468-468c-be69-e7e41de57781/server/0.log" Jul 03 20:57:38.667059 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:38.666989 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6tfkc_91a7f796-addb-4293-9e54-07cd9433cce3/manager/0.log" Jul 03 20:57:38.785207 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:38.785180 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8554cfd646-8l6wh_b1a081df-9d43-4d9b-a005-df7b6639a7e9/copy-manifests/0.log" Jul 03 20:57:38.797303 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:38.797278 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8554cfd646-8l6wh_b1a081df-9d43-4d9b-a005-df7b6639a7e9/manager/0.log" Jul 03 20:57:39.156038 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:39.156010 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-8xzxp_21b1004f-1493-4912-b664-73588801f758/postgres/0.log" Jul 03 20:57:40.585175 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:40.585144 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-r8zvb_cd8f7050-111c-4d0b-a046-9cddcc7da06a/manager/0.log" Jul 03 20:57:40.704834 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:40.704810 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-d8wbw_4abbcd18-88af-4e83-9fef-73920817cb2e/manager/0.log" Jul 03 20:57:40.940149 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:40.940121 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-8m99h_0f89b024-bb82-4a03-85e3-df0585ce2682/registry-server/0.log" Jul 03 20:57:41.070178 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:41.070149 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-s6pfk_a4f476df-78c2-468b-9d44-0ecee9539e34/manager/0.log" Jul 03 20:57:41.353284 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:41.353219 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-slzfp_54a81c57-f8f4-4dd2-af7a-bc3e3ea143ec/manager/0.log" Jul 03 20:57:41.950886 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:41.950860 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5546b876fd-gqk58_4a2d0db1-3f77-4931-b456-58ea5a1f46de/kube-auth-proxy/0.log" Jul 03 20:57:42.413338 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:42.413308 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-p4gh8_cc73801b-35f6-45d9-992c-ca9b9ca8fc00/payload-processing/0.log" Jul 03 20:57:42.990669 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:42.990642 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-sq6pj_bf8875b8-47b9-4c8e-a76f-1f9c9bc08f23/main/0.log" Jul 03 20:57:43.333726 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:43.333642 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-mpg6r_e34f6aa7-dcbd-4a0e-9246-6405bcadee0e/main/0.log" Jul 03 20:57:48.115303 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.115272 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-47n4w/must-gather-2lvc4"] Jul 03 20:57:48.118681 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.118664 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47n4w/must-gather-2lvc4" Jul 03 20:57:48.121288 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.121267 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-47n4w\"/\"default-dockercfg-44r9n\"" Jul 03 20:57:48.121417 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.121266 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-47n4w\"/\"kube-root-ca.crt\"" Jul 03 20:57:48.121417 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.121267 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-47n4w\"/\"openshift-service-ca.crt\"" Jul 03 20:57:48.135412 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.135393 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47n4w/must-gather-2lvc4"] Jul 03 20:57:48.270535 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.270495 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd66j\" (UniqueName: \"kubernetes.io/projected/3d480638-ba12-4eeb-acc4-992199918a07-kube-api-access-bd66j\") pod \"must-gather-2lvc4\" (UID: \"3d480638-ba12-4eeb-acc4-992199918a07\") " pod="openshift-must-gather-47n4w/must-gather-2lvc4" Jul 03 20:57:48.270705 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.270560 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d480638-ba12-4eeb-acc4-992199918a07-must-gather-output\") pod \"must-gather-2lvc4\" (UID: \"3d480638-ba12-4eeb-acc4-992199918a07\") " pod="openshift-must-gather-47n4w/must-gather-2lvc4" Jul 03 20:57:48.371562 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.371490 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd66j\" (UniqueName: \"kubernetes.io/projected/3d480638-ba12-4eeb-acc4-992199918a07-kube-api-access-bd66j\") pod \"must-gather-2lvc4\" (UID: \"3d480638-ba12-4eeb-acc4-992199918a07\") " pod="openshift-must-gather-47n4w/must-gather-2lvc4" Jul 03 20:57:48.371562 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.371531 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d480638-ba12-4eeb-acc4-992199918a07-must-gather-output\") pod \"must-gather-2lvc4\" (UID: \"3d480638-ba12-4eeb-acc4-992199918a07\") " pod="openshift-must-gather-47n4w/must-gather-2lvc4" Jul 03 20:57:48.371858 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.371842 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d480638-ba12-4eeb-acc4-992199918a07-must-gather-output\") pod \"must-gather-2lvc4\" (UID: \"3d480638-ba12-4eeb-acc4-992199918a07\") " pod="openshift-must-gather-47n4w/must-gather-2lvc4" Jul 03 20:57:48.380079 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.380052 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd66j\" (UniqueName: \"kubernetes.io/projected/3d480638-ba12-4eeb-acc4-992199918a07-kube-api-access-bd66j\") pod \"must-gather-2lvc4\" (UID: \"3d480638-ba12-4eeb-acc4-992199918a07\") " pod="openshift-must-gather-47n4w/must-gather-2lvc4" Jul 03 20:57:48.428676 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.428646 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47n4w/must-gather-2lvc4" Jul 03 20:57:48.566693 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.566666 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47n4w/must-gather-2lvc4"] Jul 03 20:57:48.568666 ip-10-0-139-68 kubenswrapper[2582]: W0703 20:57:48.568641 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d480638_ba12_4eeb_acc4_992199918a07.slice/crio-403b03bcdfe5143cb4875fae7df53faf53203a4c7be7e8ca7917775a7b958b33 WatchSource:0}: Error finding container 403b03bcdfe5143cb4875fae7df53faf53203a4c7be7e8ca7917775a7b958b33: Status 404 returned error can't find the container with id 403b03bcdfe5143cb4875fae7df53faf53203a4c7be7e8ca7917775a7b958b33 Jul 03 20:57:48.570891 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.570876 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 20:57:48.769241 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:48.769204 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47n4w/must-gather-2lvc4" event={"ID":"3d480638-ba12-4eeb-acc4-992199918a07","Type":"ContainerStarted","Data":"403b03bcdfe5143cb4875fae7df53faf53203a4c7be7e8ca7917775a7b958b33"} Jul 03 20:57:49.785779 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:49.785717 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47n4w/must-gather-2lvc4" event={"ID":"3d480638-ba12-4eeb-acc4-992199918a07","Type":"ContainerStarted","Data":"56f1d70e220d6938022beedf59ea73f80cdabdc77fffd53e94fd5d46e8755e07"} Jul 03 20:57:49.786223 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:49.785795 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47n4w/must-gather-2lvc4" event={"ID":"3d480638-ba12-4eeb-acc4-992199918a07","Type":"ContainerStarted","Data":"771b05ce980d1d8ed773220b9462767a8cbb65acaf80409d0175fc5f9fd4beb8"} Jul 03 20:57:49.803701 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:49.803473 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-47n4w/must-gather-2lvc4" podStartSLOduration=1.109195712 podStartE2EDuration="1.803451871s" totalImagesPullingTime="694.256159ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:57:49.800553728 +0000 UTC m=+1536.258826170" watchObservedRunningTime="2026-07-03 20:57:49.803451871 +0000 UTC m=+1536.261724311" Jul 03 20:57:50.802702 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:50.802673 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-chq7m_50e502c5-dd61-4c6f-9bcf-6de839bf5fbb/global-pull-secret-syncer/0.log" Jul 03 20:57:50.948467 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:50.948434 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ng2rm_ee651638-933f-4b70-a0b4-08438004b94d/konnectivity-agent/0.log" Jul 03 20:57:51.009616 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:51.009586 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-68.ec2.internal_6deb43b1e3d0667ae47de4e2350bbbcb/haproxy/0.log" Jul 03 20:57:55.682018 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:55.681990 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-r8zvb_cd8f7050-111c-4d0b-a046-9cddcc7da06a/manager/0.log" Jul 03 20:57:55.706535 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:55.706465 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-d8wbw_4abbcd18-88af-4e83-9fef-73920817cb2e/manager/0.log" Jul 03 20:57:55.761089 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:55.761058 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-8m99h_0f89b024-bb82-4a03-85e3-df0585ce2682/registry-server/0.log" Jul 03 20:57:55.819048 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:55.819018 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-s6pfk_a4f476df-78c2-468b-9d44-0ecee9539e34/manager/0.log" Jul 03 20:57:55.897654 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:55.897620 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-slzfp_54a81c57-f8f4-4dd2-af7a-bc3e3ea143ec/manager/0.log" Jul 03 20:57:57.976192 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:57.976157 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qhn7n_be7d78b0-7c53-41cf-805e-65c97e3b0ec2/node-exporter/0.log" Jul 03 20:57:58.004354 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:58.004328 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qhn7n_be7d78b0-7c53-41cf-805e-65c97e3b0ec2/kube-rbac-proxy/0.log" Jul 03 20:57:58.026036 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:57:58.025946 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qhn7n_be7d78b0-7c53-41cf-805e-65c97e3b0ec2/init-textfile/0.log" Jul 03 20:58:01.140103 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.140056 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx"] Jul 03 20:58:01.149266 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.149238 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.150462 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.150426 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx"] Jul 03 20:58:01.190171 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.190129 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e82a1c1b-d428-4702-9b1c-0002339a7452-proc\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.190500 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.190480 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e82a1c1b-d428-4702-9b1c-0002339a7452-podres\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.190695 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.190678 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqkht\" (UniqueName: \"kubernetes.io/projected/e82a1c1b-d428-4702-9b1c-0002339a7452-kube-api-access-lqkht\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.190850 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.190833 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e82a1c1b-d428-4702-9b1c-0002339a7452-lib-modules\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.190994 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.190978 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e82a1c1b-d428-4702-9b1c-0002339a7452-sys\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.291579 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.291548 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqkht\" (UniqueName: \"kubernetes.io/projected/e82a1c1b-d428-4702-9b1c-0002339a7452-kube-api-access-lqkht\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.291785 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.291594 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e82a1c1b-d428-4702-9b1c-0002339a7452-lib-modules\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.291785 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.291635 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e82a1c1b-d428-4702-9b1c-0002339a7452-sys\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.291785 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.291663 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e82a1c1b-d428-4702-9b1c-0002339a7452-proc\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.291785 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.291722 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e82a1c1b-d428-4702-9b1c-0002339a7452-podres\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.291980 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.291793 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e82a1c1b-d428-4702-9b1c-0002339a7452-sys\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.291980 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.291846 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e82a1c1b-d428-4702-9b1c-0002339a7452-lib-modules\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.291980 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.291871 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e82a1c1b-d428-4702-9b1c-0002339a7452-proc\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.291980 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.291884 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e82a1c1b-d428-4702-9b1c-0002339a7452-podres\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.299279 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.299252 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqkht\" (UniqueName: \"kubernetes.io/projected/e82a1c1b-d428-4702-9b1c-0002339a7452-kube-api-access-lqkht\") pod \"perf-node-gather-daemonset-lpwpx\" (UID: \"e82a1c1b-d428-4702-9b1c-0002339a7452\") " pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.464721 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.464686 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.636268 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.636149 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx"] Jul 03 20:58:01.881632 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.881518 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" event={"ID":"e82a1c1b-d428-4702-9b1c-0002339a7452","Type":"ContainerStarted","Data":"b62f3a38713d3b6351582a47cfbaf13563b2b4fc4b54f7a6ed0625654ca4e6bb"} Jul 03 20:58:01.881632 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.881567 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" event={"ID":"e82a1c1b-d428-4702-9b1c-0002339a7452","Type":"ContainerStarted","Data":"4d5c2f50505e0f2f140ac2bba177dd7b6c070099ffff3ba150f74876e58e9b9e"} Jul 03 20:58:01.881839 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.881636 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:01.898318 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:01.898263 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" podStartSLOduration=0.898242586 podStartE2EDuration="898.242586ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:58:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:58:01.896503969 +0000 UTC m=+1548.354776409" watchObservedRunningTime="2026-07-03 20:58:01.898242586 +0000 UTC m=+1548.356515025" Jul 03 20:58:02.945760 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:02.945710 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rx9p2_0d42b917-3c23-4fa8-8d71-ff9742a87e12/dns/0.log" Jul 03 20:58:02.965282 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:02.965252 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rx9p2_0d42b917-3c23-4fa8-8d71-ff9742a87e12/kube-rbac-proxy/0.log" Jul 03 20:58:03.027602 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:03.027570 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4lfd9_e148c3c4-214b-4ed2-acea-69d3a0fe10b2/dns-node-resolver/0.log" Jul 03 20:58:03.608461 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:03.608418 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-684848dfc7-kqrbk_1097afbf-4798-4695-8714-8f343b8646fc/registry/0.log" Jul 03 20:58:03.627730 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:03.627706 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bs4f9_f6a8fc1e-c013-45e0-a570-97d2e74f7feb/node-ca/0.log" Jul 03 20:58:04.623788 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:04.623737 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5546b876fd-gqk58_4a2d0db1-3f77-4931-b456-58ea5a1f46de/kube-auth-proxy/0.log" Jul 03 20:58:04.747079 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:04.747054 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-p4gh8_cc73801b-35f6-45d9-992c-ca9b9ca8fc00/payload-processing/0.log" Jul 03 20:58:05.374893 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:05.374865 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kwwh4_cb443c07-c7e6-42e6-9d95-baed044fcdf9/serve-healthcheck-canary/0.log" Jul 03 20:58:06.005429 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:06.005403 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v7j67_2004c927-d12e-4dc3-9c28-01d6e488fc0e/kube-rbac-proxy/0.log" Jul 03 20:58:06.024624 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:06.024598 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v7j67_2004c927-d12e-4dc3-9c28-01d6e488fc0e/exporter/0.log" Jul 03 20:58:06.045700 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:06.045673 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v7j67_2004c927-d12e-4dc3-9c28-01d6e488fc0e/extractor/0.log" Jul 03 20:58:07.898554 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:07.898464 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-47n4w/perf-node-gather-daemonset-lpwpx" Jul 03 20:58:08.100872 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:08.100843 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tvcsp_b1a66c17-ccf0-4802-86b0-b16795c603a4/manager/0.log" Jul 03 20:58:08.135634 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:08.135587 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-k27gj_bb5c5346-91f7-4f20-b3bf-e2beb1aec43b/manager/0.log" Jul 03 20:58:08.176155 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:08.176082 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-594b7cb88d-s82mn_5e98bdf4-940b-40f3-97ab-b3d113e96590/maas-api/0.log" Jul 03 20:58:08.241141 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:08.241114 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-746df7898c-ts9ms_85950ae0-9eaf-435e-b19d-14c4c6172384/manager/0.log" Jul 03 20:58:08.260579 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:08.260551 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-wd9c5_e0ed9b90-a468-468c-be69-e7e41de57781/server/0.log" Jul 03 20:58:08.312042 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:08.312015 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6tfkc_91a7f796-addb-4293-9e54-07cd9433cce3/manager/0.log" Jul 03 20:58:08.370706 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:08.370668 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8554cfd646-8l6wh_b1a081df-9d43-4d9b-a005-df7b6639a7e9/manager/0.log" Jul 03 20:58:08.386164 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:08.386137 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8554cfd646-8l6wh_b1a081df-9d43-4d9b-a005-df7b6639a7e9/copy-manifests/0.log" Jul 03 20:58:08.506142 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:08.506064 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-8xzxp_21b1004f-1493-4912-b664-73588801f758/postgres/0.log" Jul 03 20:58:11.443092 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:11.443065 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5546b876fd-gqk58_4a2d0db1-3f77-4931-b456-58ea5a1f46de/kube-auth-proxy/0.log" Jul 03 20:58:11.529112 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:11.529080 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-p4gh8_cc73801b-35f6-45d9-992c-ca9b9ca8fc00/payload-processing/0.log" Jul 03 20:58:12.681627 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:12.681590 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5546b876fd-gqk58_4a2d0db1-3f77-4931-b456-58ea5a1f46de/kube-auth-proxy/0.log" Jul 03 20:58:12.780717 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:12.780684 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-p4gh8_cc73801b-35f6-45d9-992c-ca9b9ca8fc00/payload-processing/0.log" Jul 03 20:58:14.981952 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:14.981915 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-xfk8t_a7456447-e65b-4775-bec1-822fb72392c8/migrator/0.log" Jul 03 20:58:15.003148 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:15.003121 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-xfk8t_a7456447-e65b-4775-bec1-822fb72392c8/graceful-termination/0.log" Jul 03 20:58:16.365495 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:16.365463 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7jd92_5609fd01-5426-4ff6-97f5-a3d3367e85f0/kube-multus-additional-cni-plugins/0.log" Jul 03 20:58:16.385802 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:16.385772 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7jd92_5609fd01-5426-4ff6-97f5-a3d3367e85f0/egress-router-binary-copy/0.log" Jul 03 20:58:16.404266 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:16.404241 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7jd92_5609fd01-5426-4ff6-97f5-a3d3367e85f0/cni-plugins/0.log" Jul 03 20:58:16.424289 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:16.424268 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7jd92_5609fd01-5426-4ff6-97f5-a3d3367e85f0/bond-cni-plugin/0.log" Jul 03 20:58:16.443763 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:16.443732 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7jd92_5609fd01-5426-4ff6-97f5-a3d3367e85f0/routeoverride-cni/0.log" Jul 03 20:58:16.464828 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:16.464806 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7jd92_5609fd01-5426-4ff6-97f5-a3d3367e85f0/whereabouts-cni-bincopy/0.log" Jul 03 20:58:16.488431 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:16.488409 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7jd92_5609fd01-5426-4ff6-97f5-a3d3367e85f0/whereabouts-cni/0.log" Jul 03 20:58:16.812252 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:16.812221 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cztbp_4483b180-eb1d-488d-a652-4f12c1e41407/kube-multus/0.log" Jul 03 20:58:16.969718 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:16.969691 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s5vq9_499e1abf-db52-4ca2-9ea6-208a405209f3/network-metrics-daemon/0.log" Jul 03 20:58:16.989163 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:16.989140 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s5vq9_499e1abf-db52-4ca2-9ea6-208a405209f3/kube-rbac-proxy/0.log" Jul 03 20:58:18.011657 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:18.011583 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8gc9_f8c1736d-e1e0-4fed-979c-4039a3b4c087/ovn-controller/0.log" Jul 03 20:58:18.026969 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:18.026947 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8gc9_f8c1736d-e1e0-4fed-979c-4039a3b4c087/ovn-acl-logging/0.log" Jul 03 20:58:18.048079 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:18.048046 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8gc9_f8c1736d-e1e0-4fed-979c-4039a3b4c087/kube-rbac-proxy-node/0.log" Jul 03 20:58:18.066669 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:18.066646 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8gc9_f8c1736d-e1e0-4fed-979c-4039a3b4c087/kube-rbac-proxy-ovn-metrics/0.log" Jul 03 20:58:18.083967 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:18.083940 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8gc9_f8c1736d-e1e0-4fed-979c-4039a3b4c087/northd/0.log" Jul 03 20:58:18.102541 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:18.102522 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8gc9_f8c1736d-e1e0-4fed-979c-4039a3b4c087/nbdb/0.log" Jul 03 20:58:18.121534 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:18.121516 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8gc9_f8c1736d-e1e0-4fed-979c-4039a3b4c087/sbdb/0.log" Jul 03 20:58:18.163453 ip-10-0-139-68 kubenswrapper[2582]: I0703 20:58:18.163407 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8gc9_f8c1736d-e1e0-4fed-979c-4039a3b4c087/ovnkube-controller/0.log"