Jul 09 14:18:40.019646 ip-10-0-136-93 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 14:18:40.019661 ip-10-0-136-93 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 14:18:40.019671 ip-10-0-136-93 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 14:18:40.019992 ip-10-0-136-93 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 14:18:51.459413 ip-10-0-136-93 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 09 14:18:51.459428 ip-10-0-136-93 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e2e29e58af1a40f0b16f3d0cfa91738d -- Jul 09 14:20:49.529156 ip-10-0-136-93 systemd[1]: Starting Kubernetes Kubelet... Jul 09 14:20:49.844929 ip-10-0-136-93 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 14:20:49.844929 ip-10-0-136-93 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 14:20:49.844929 ip-10-0-136-93 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 14:20:49.844929 ip-10-0-136-93 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 14:20:49.844929 ip-10-0-136-93 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 14:20:49.846780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.846688 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 14:20:49.851222 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851205 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:20:49.851222 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851221 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851225 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851228 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851232 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851234 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851238 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851241 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851243 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851246 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851249 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851255 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851258 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851261 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851263 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851266 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851270 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851274 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851277 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851280 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:20:49.851288 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851283 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851286 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851289 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851292 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851295 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851298 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851300 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851303 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851305 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851308 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851311 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851313 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851316 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851318 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851320 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851323 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851325 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851328 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851331 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851333 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:20:49.851790 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851336 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851338 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851340 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851343 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851346 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851348 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851351 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851353 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851355 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851358 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851361 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851364 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851368 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851370 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851373 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851376 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851379 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851381 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851384 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851395 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:20:49.852556 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851398 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851402 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851406 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851409 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851412 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851414 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851417 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851419 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851422 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851424 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851426 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851429 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851432 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851434 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851437 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851439 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851441 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851444 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851447 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:20:49.853123 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851449 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851453 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851455 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851458 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851461 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851465 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851469 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851473 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851476 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851478 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851481 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851483 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851486 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851488 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851491 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851494 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851496 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851499 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851502 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:20:49.853583 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.851504 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853188 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853200 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853203 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853207 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853210 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853214 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853217 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853219 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853222 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853225 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853228 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853231 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853233 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853236 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853239 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853241 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853244 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853246 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853249 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:20:49.854055 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853252 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853254 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853257 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853259 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853262 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853264 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853267 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853269 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853272 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853274 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853277 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853279 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853283 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853288 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853291 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853295 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853298 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853300 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853303 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:20:49.854546 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853305 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853308 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853311 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853313 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853315 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853319 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853324 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853327 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853330 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853333 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853335 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853338 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853341 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853344 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853346 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853349 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853351 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853354 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853356 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:20:49.855025 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853359 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853361 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853364 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853366 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853369 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853371 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853373 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853376 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853380 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853382 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853385 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853388 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853390 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853393 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853396 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853398 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853401 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853403 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853406 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853408 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:20:49.855498 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853410 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853413 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853415 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853418 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853420 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853422 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853426 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853429 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853433 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853436 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853439 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853441 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853444 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853447 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853450 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853452 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853455 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853457 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853461 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:20:49.856008 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853464 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853467 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853470 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853887 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853894 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853898 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853901 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853903 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853906 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853909 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853911 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853914 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853916 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853919 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853921 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853924 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853927 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853929 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853931 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853935 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:20:49.856478 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853937 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853940 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853943 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853946 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853948 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853951 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853953 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853957 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853961 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853964 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853967 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853970 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853972 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853975 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853978 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853980 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853982 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853985 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853988 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853990 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:20:49.856987 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853992 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853995 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.853998 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854001 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854004 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854008 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854011 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854013 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854016 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854018 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854020 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854023 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854026 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854028 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854030 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854033 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854035 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854038 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854040 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:20:49.857463 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854043 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854045 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854048 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854052 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854056 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854060 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854063 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854066 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854069 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854072 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854075 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854078 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854081 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854083 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854085 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854088 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854091 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854094 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854096 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:20:49.857963 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854098 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854101 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854103 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854106 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854108 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854110 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854113 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854116 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854118 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854121 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854123 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854125 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854128 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854130 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854133 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854135 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854138 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854141 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854144 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854146 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:20:49.858406 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854149 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854151 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854153 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854156 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854231 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854238 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854245 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854249 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854253 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854256 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854260 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854265 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854268 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854271 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854274 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854277 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854280 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854283 2582 flags.go:64] FLAG: --cgroup-root="" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854286 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854289 2582 flags.go:64] FLAG: --client-ca-file="" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854293 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854296 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854300 2582 flags.go:64] FLAG: --cluster-domain="" Jul 09 14:20:49.858910 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854302 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854305 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854308 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854312 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854316 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854319 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854325 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854329 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854332 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854334 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854337 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854341 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854344 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854348 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854351 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854354 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854357 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854360 2582 flags.go:64] FLAG: --enable-server="true" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854363 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854367 2582 flags.go:64] FLAG: --event-burst="100" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854370 2582 flags.go:64] FLAG: --event-qps="50" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854373 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854376 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854379 2582 flags.go:64] FLAG: --eviction-hard="" Jul 09 14:20:49.859465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854383 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854385 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854388 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854391 2582 flags.go:64] FLAG: --eviction-soft="" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854394 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854397 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854403 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854406 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854409 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854411 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854414 2582 flags.go:64] FLAG: --feature-gates="" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854418 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854421 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854424 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854429 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854432 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854434 2582 flags.go:64] FLAG: --help="false" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854437 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-136-93.ec2.internal" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854441 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854444 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854447 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854450 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854453 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854457 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 14:20:49.860070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854459 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854462 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854465 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854467 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854470 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854473 2582 flags.go:64] FLAG: --kube-reserved="" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854476 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854479 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854481 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854484 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854487 2582 flags.go:64] FLAG: --lock-file="" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854489 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854492 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854495 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854500 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854504 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854507 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854509 2582 flags.go:64] FLAG: --logging-format="text" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854512 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854515 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854518 2582 flags.go:64] FLAG: --manifest-url="" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854521 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854527 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854530 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854534 2582 flags.go:64] FLAG: --max-pods="110" Jul 09 14:20:49.860678 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854536 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854539 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854542 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854545 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854548 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854551 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854554 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854561 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854564 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854567 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854570 2582 flags.go:64] FLAG: --pod-cidr="" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854572 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854578 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854580 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854583 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854586 2582 flags.go:64] FLAG: --port="10250" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854589 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854592 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09487c9e2d3d6c7c1" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854595 2582 flags.go:64] FLAG: --qos-reserved="" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854598 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854601 2582 flags.go:64] FLAG: --register-node="true" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854603 2582 flags.go:64] FLAG: --register-with-taints="" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854607 2582 flags.go:64] FLAG: --registry-burst="10" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854610 2582 flags.go:64] FLAG: --registry-qps="5" Jul 09 14:20:49.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854613 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854615 2582 flags.go:64] FLAG: --reserved-memory="" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854623 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854626 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854629 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854633 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854636 2582 flags.go:64] FLAG: --runonce="false" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854639 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854642 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854645 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854647 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854650 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854653 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854656 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854659 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854661 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854664 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854667 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854669 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854672 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854675 2582 flags.go:64] FLAG: --system-cgroups="" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854677 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854682 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854685 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854688 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 14:20:49.861960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854691 2582 flags.go:64] FLAG: --tls-min-version="" Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854694 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854697 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854700 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854702 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854705 2582 flags.go:64] FLAG: --v="2" Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854709 2582 flags.go:64] FLAG: --version="false" Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854714 2582 flags.go:64] FLAG: --vmodule="" Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854717 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.854721 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854844 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854848 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854853 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854856 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854860 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854863 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854866 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854882 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854885 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854888 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854890 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854893 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:20:49.862570 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854896 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854898 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854901 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854904 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854906 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854909 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854911 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854914 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854917 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854919 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854921 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854924 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854926 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854928 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854931 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854933 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854936 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854938 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854941 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854944 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:20:49.863125 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854946 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854949 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854953 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854955 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854958 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854960 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854962 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854965 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854967 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854970 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854972 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854976 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854979 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854982 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854984 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854988 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854992 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854994 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.854997 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:20:49.863624 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855000 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855003 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855006 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855013 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855016 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855019 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855022 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855024 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855027 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855030 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855034 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855037 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855040 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855043 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855045 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855049 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855051 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855054 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855057 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855059 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:20:49.864115 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855061 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855064 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855067 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855070 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855074 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855076 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855079 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855081 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855084 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855086 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855089 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855091 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855094 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855096 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855099 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855101 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855103 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855106 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855109 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855111 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:20:49.864598 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855114 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855116 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855120 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855122 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855125 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855128 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855131 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.855135 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.855714 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.862756 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.862774 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.862794 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.862799 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862854 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862859 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:20:49.865202 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862861 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862864 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862867 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862886 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862889 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862892 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862895 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862898 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862900 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862903 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862906 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862908 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862911 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862913 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862916 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862918 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862921 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862923 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862926 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862928 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:20:49.865584 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862930 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862933 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862937 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862940 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862943 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862945 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862950 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862954 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862958 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862961 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862964 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862967 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862970 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862973 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862975 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862978 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862980 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862983 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862986 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862988 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:20:49.866097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862991 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862993 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862995 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.862998 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863000 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863003 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863006 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863008 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863011 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863013 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863015 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863018 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863020 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863023 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863025 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863029 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863031 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863034 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863036 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863039 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:20:49.866557 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863042 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863045 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863047 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863050 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863052 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863055 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863058 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863062 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863065 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863067 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863070 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863072 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863075 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863078 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863080 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863082 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863085 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863088 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863090 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:20:49.867093 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863093 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863095 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863097 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863100 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863103 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863105 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863108 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863110 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863113 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863116 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863119 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863122 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863124 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863127 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863129 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863133 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863137 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:20:49.867567 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863140 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.863145 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863254 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863259 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863261 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863264 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863267 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863270 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863273 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863276 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863279 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863281 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863283 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863286 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863288 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 14:20:49.868041 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863291 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863294 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863296 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863299 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863301 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863304 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863306 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863309 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863311 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863314 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863317 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863319 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863322 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863324 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863327 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863329 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863331 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863334 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863337 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 14:20:49.868408 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863339 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863341 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863344 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863346 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863349 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863351 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863353 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863356 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863358 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863361 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863363 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863366 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863368 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863371 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863373 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863376 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863378 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863380 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863383 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863392 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 14:20:49.868908 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863396 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863398 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863401 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863403 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863407 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863409 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863412 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863414 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863417 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863420 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863422 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863424 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863427 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863429 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863431 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863435 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863438 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863441 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863444 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 14:20:49.869398 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863447 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863449 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863452 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863455 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863458 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863461 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863465 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863469 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863472 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863476 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863479 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863481 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863484 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863487 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863489 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863495 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863498 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863500 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863503 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863505 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 14:20:49.869863 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863508 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 14:20:49.870514 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863510 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 14:20:49.870514 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863512 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 14:20:49.870514 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863515 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 14:20:49.870514 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863517 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 14:20:49.870514 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863519 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 14:20:49.870514 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863522 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 14:20:49.870514 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:49.863524 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 14:20:49.870514 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.863529 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 14:20:49.870514 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.863644 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 14:20:49.870514 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.865551 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 14:20:49.870514 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.866690 2582 server.go:1019] "Starting client certificate rotation" Jul 09 14:20:49.870514 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.866783 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 14:20:49.870514 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.866823 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 14:20:49.870848 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.868663 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 14:20:49.870848 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.869787 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 14:20:49.878436 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.878415 2582 log.go:25] "Validated CRI v1 runtime API" Jul 09 14:20:49.880370 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.880353 2582 log.go:25] "Validated CRI v1 image API" Jul 09 14:20:49.881164 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.881142 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 14:20:49.883243 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.883214 2582 fs.go:135] Filesystem UUIDs: map[3148811f-f52d-470f-9e1f-ee9576be7cfa:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7ab8b364-db17-40ac-a7d6-a05799d81f00:/dev/nvme0n1p3] Jul 09 14:20:49.883325 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.883241 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 14:20:49.889063 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.888948 2582 manager.go:216] Machine: {Timestamp:2026-07-09 14:20:49.887911459 +0000 UTC m=+0.254005835 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091317 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22292aacfae078846b51800ff835e1 SystemUUID:ec22292a-acfa-e078-846b-51800ff835e1 BootID:e2e29e58-af1a-40f0-b16f-3d0cfa91738d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8c:63:f6:51:6f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8c:63:f6:51:6f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:c9:51:b0:53:48 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 14:20:49.889063 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.889049 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 14:20:49.889227 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.889161 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 14:20:49.892445 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.892421 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 14:20:49.892616 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.892448 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-93.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 14:20:49.893061 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.893051 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 14:20:49.893098 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.893064 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 14:20:49.893098 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.893071 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 14:20:49.893637 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.893624 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 14:20:49.894907 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.894897 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 14:20:49.896681 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.896663 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 14:20:49.897484 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.897472 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 09 14:20:49.897603 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.897594 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 14:20:49.897634 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.897625 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 09 14:20:49.897661 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.897635 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 14:20:49.897661 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.897650 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 14:20:49.897661 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.897659 2582 kubelet.go:398] "Adding apiserver pod source" Jul 09 14:20:49.897753 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.897667 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 14:20:49.898047 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.898036 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 14:20:49.898097 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.898053 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 14:20:49.899783 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.899766 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 09 14:20:49.901634 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.901620 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 14:20:49.901686 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.901651 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 14:20:49.901816 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.901806 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 14:20:49.901851 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.901821 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 14:20:49.901851 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.901827 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 14:20:49.901851 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.901833 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 14:20:49.901851 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.901841 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 14:20:49.901851 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.901847 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 14:20:49.902210 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.902201 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 14:20:49.902243 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.902211 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 14:20:49.902243 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.902218 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 14:20:49.902243 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.902225 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 14:20:49.902243 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.902241 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 14:20:49.902340 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.902249 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 14:20:49.903334 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.903322 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 14:20:49.903414 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.903402 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 14:20:49.906796 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.906782 2582 server.go:1268] "Started kubelet" Jul 09 14:20:49.906922 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.906891 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 14:20:49.907214 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.906865 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 14:20:49.907214 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.907104 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 14:20:49.909703 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.909671 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 14:20:49.910051 ip-10-0-136-93 systemd[1]: Started Kubernetes Kubelet. Jul 09 14:20:49.912548 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.911925 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 09 14:20:49.912548 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:49.912205 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 14:20:49.914404 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.914386 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 14:20:49.914470 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:49.914395 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 14:20:49.914564 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:49.914546 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 14:20:49.915072 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:49.914328 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-93.ec2.internal.18c0a4fae3a4ea33 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-93.ec2.internal,UID:ip-10-0-136-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-93.ec2.internal,},FirstTimestamp:2026-07-09 14:20:49.906731571 +0000 UTC m=+0.272825947,LastTimestamp:2026-07-09 14:20:49.906731571 +0000 UTC m=+0.272825947,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-93.ec2.internal,}" Jul 09 14:20:49.917201 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.917182 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 14:20:49.917293 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.917200 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 14:20:49.917339 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:49.917304 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-93.ec2.internal\" not found" Jul 09 14:20:49.917769 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.917749 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 14:20:49.917845 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.917787 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 14:20:49.917845 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.917748 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 14:20:49.917956 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.917890 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 09 14:20:49.917956 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.917900 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 09 14:20:49.918183 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.918128 2582 factory.go:55] Registering systemd factory Jul 09 14:20:49.918183 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.918167 2582 factory.go:223] Registration of the systemd container factory successfully Jul 09 14:20:49.918387 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.918375 2582 factory.go:153] Registering CRI-O factory Jul 09 14:20:49.918438 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.918389 2582 factory.go:223] Registration of the crio container factory successfully Jul 09 14:20:49.918481 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.918441 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 14:20:49.918481 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.918465 2582 factory.go:103] Registering Raw factory Jul 09 14:20:49.918576 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.918482 2582 manager.go:1196] Started watching for new ooms in manager Jul 09 14:20:49.919112 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.919096 2582 manager.go:349] Starting recovery of all containers Jul 09 14:20:49.928433 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.928270 2582 manager.go:354] Recovery completed Jul 09 14:20:49.928599 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.928581 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w8khq" Jul 09 14:20:49.930105 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:49.930082 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 14:20:49.930213 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:49.930136 2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 14:20:49.933124 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.933113 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:20:49.935063 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.935048 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:20:49.935121 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.935079 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:20:49.935121 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.935090 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:20:49.935677 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.935664 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:20:49.935748 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.935663 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 14:20:49.935748 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.935721 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 14:20:49.935748 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.935741 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 09 14:20:49.936341 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.936324 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:20:49.936415 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.936354 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:20:49.936415 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.936367 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:20:49.936755 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.936744 2582 policy_none.go:49] "None policy: Start" Jul 09 14:20:49.936799 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.936758 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 14:20:49.936799 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.936768 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 14:20:49.937827 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.937814 2582 policy_none.go:47] "Start" Jul 09 14:20:49.937827 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.937814 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w8khq" Jul 09 14:20:49.978962 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.978946 2582 manager.go:337] "Starting Device Plugin manager" Jul 09 14:20:49.979080 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:49.978982 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 14:20:49.979080 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.978991 2582 server.go:87] "Starting device plugin registration server" Jul 09 14:20:49.979202 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.979189 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 14:20:49.979244 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.979206 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 14:20:49.979314 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.979296 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 14:20:49.979401 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.979382 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 14:20:49.979450 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:49.979402 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 14:20:49.980627 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:49.980609 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 14:20:49.980737 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:49.980657 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-93.ec2.internal\" not found" Jul 09 14:20:50.029572 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.029546 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 14:20:50.031713 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.031009 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 14:20:50.031713 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.031054 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 14:20:50.031713 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.031081 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 14:20:50.031713 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:50.031120 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 14:20:50.033326 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.033304 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:20:50.079441 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.079416 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:20:50.080314 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.080299 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:20:50.080407 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.080332 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:20:50.080407 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.080346 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:20:50.080407 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.080376 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.086633 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.086619 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.086708 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:50.086640 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-93.ec2.internal\": node \"ip-10-0-136-93.ec2.internal\" not found" Jul 09 14:20:50.100694 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:50.100639 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-93.ec2.internal\" not found" Jul 09 14:20:50.131715 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.131692 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-93.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal"] Jul 09 14:20:50.131809 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.131760 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:20:50.132654 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.132639 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:20:50.132731 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.132673 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:20:50.132731 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.132684 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:20:50.134506 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.134492 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:20:50.134619 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.134605 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.134676 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.134640 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:20:50.135175 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.135159 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:20:50.135256 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.135185 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:20:50.135256 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.135198 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:20:50.135256 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.135164 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:20:50.135256 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.135234 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:20:50.135256 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.135246 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:20:50.136990 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.136975 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.137058 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.137002 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 14:20:50.137682 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.137666 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasSufficientMemory" Jul 09 14:20:50.137749 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.137693 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 14:20:50.137749 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.137706 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeHasSufficientPID" Jul 09 14:20:50.155475 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:50.155452 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-93.ec2.internal\" not found" node="ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.159350 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:50.159333 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-93.ec2.internal\" not found" node="ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.201223 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:50.201199 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-93.ec2.internal\" not found" Jul 09 14:20:50.301441 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:50.301414 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-93.ec2.internal\" not found" Jul 09 14:20:50.319771 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.319750 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0b2614806c45fdfd93e51f60f22bdd4c-config\") pod \"kube-apiserver-proxy-ip-10-0-136-93.ec2.internal\" (UID: \"0b2614806c45fdfd93e51f60f22bdd4c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.319830 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.319777 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a427c6d8749af06dbad4543acc38acf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal\" (UID: \"0a427c6d8749af06dbad4543acc38acf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.319830 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.319794 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a427c6d8749af06dbad4543acc38acf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal\" (UID: \"0a427c6d8749af06dbad4543acc38acf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.402205 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:50.402150 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-93.ec2.internal\" not found" Jul 09 14:20:50.420480 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.420457 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0b2614806c45fdfd93e51f60f22bdd4c-config\") pod \"kube-apiserver-proxy-ip-10-0-136-93.ec2.internal\" (UID: \"0b2614806c45fdfd93e51f60f22bdd4c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.420561 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.420490 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a427c6d8749af06dbad4543acc38acf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal\" (UID: \"0a427c6d8749af06dbad4543acc38acf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.420561 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.420513 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a427c6d8749af06dbad4543acc38acf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal\" (UID: \"0a427c6d8749af06dbad4543acc38acf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.420652 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.420563 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a427c6d8749af06dbad4543acc38acf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal\" (UID: \"0a427c6d8749af06dbad4543acc38acf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.420652 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.420568 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0b2614806c45fdfd93e51f60f22bdd4c-config\") pod \"kube-apiserver-proxy-ip-10-0-136-93.ec2.internal\" (UID: \"0b2614806c45fdfd93e51f60f22bdd4c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.420652 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.420563 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a427c6d8749af06dbad4543acc38acf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal\" (UID: \"0a427c6d8749af06dbad4543acc38acf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.457627 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.457606 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.462281 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.462171 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.502523 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:50.502491 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-93.ec2.internal\" not found" Jul 09 14:20:50.603079 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:50.603036 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-93.ec2.internal\" not found" Jul 09 14:20:50.703929 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:50.703826 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-93.ec2.internal\" not found" Jul 09 14:20:50.742325 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.742300 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:20:50.818310 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.818279 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.830458 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.830430 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 14:20:50.832063 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.832049 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal" Jul 09 14:20:50.845685 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.845668 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 14:20:50.866680 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.866653 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 14:20:50.866797 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.866778 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 14:20:50.866843 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.866816 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 14:20:50.899430 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.899393 2582 apiserver.go:52] "Watching apiserver" Jul 09 14:20:50.917953 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.917924 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 14:20:50.919119 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.919100 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 14:20:50.920775 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.920756 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-bjpmw","openshift-ovn-kubernetes/ovnkube-node-l5qkd","kube-system/konnectivity-agent-jc75x","kube-system/kube-apiserver-proxy-ip-10-0-136-93.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss","openshift-cluster-node-tuning-operator/tuned-m8tlt","openshift-multus/multus-additional-cni-plugins-dz8cc","openshift-multus/network-metrics-daemon-vkk5f","openshift-network-diagnostics/network-check-target-kjc84","openshift-network-operator/iptables-alerter-9x98q","kube-system/global-pull-secret-syncer-8mtbj","openshift-dns/node-resolver-n9hkg","openshift-image-registry/node-ca-ffxnv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal"] Jul 09 14:20:50.923209 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.923193 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9x98q" Jul 09 14:20:50.925195 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.925175 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ffxnv" Jul 09 14:20:50.925647 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.925627 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wq4mm\"" Jul 09 14:20:50.925907 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.925888 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 14:20:50.926007 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.925937 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:20:50.926072 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.926013 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 14:20:50.927180 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.927162 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jc75x" Jul 09 14:20:50.927482 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.927465 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 14:20:50.927560 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.927546 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2q8fs\"" Jul 09 14:20:50.927623 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.927582 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 14:20:50.927682 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.927624 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 14:20:50.929220 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.929201 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:20:50.929310 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:50.929289 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:20:50.929514 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.929496 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 14:20:50.929975 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.929939 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qvqrj\"" Jul 09 14:20:50.930048 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.930019 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 14:20:50.931396 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.931379 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:50.932372 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.932351 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 14:20:50.933503 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.933487 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:50.933897 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.933881 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 14:20:50.934003 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.933902 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 14:20:50.934003 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.933920 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-85q8v\"" Jul 09 14:20:50.934003 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.933907 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 14:20:50.935694 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.935678 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-59z6z\"" Jul 09 14:20:50.936384 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.935873 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 14:20:50.936384 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.935954 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:20:50.936384 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:50.936055 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:20:50.936557 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.936495 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:20:50.938566 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.938552 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:50.940697 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.940667 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 14:15:50 +0000 UTC" deadline="2028-03-26 22:15:05.268526502 +0000 UTC" Jul 09 14:20:50.940771 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.940698 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15031h54m14.327833088s" Jul 09 14:20:50.940809 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.940772 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 14:20:50.940931 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.940904 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:50.941060 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.941018 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 14:20:50.941060 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.941033 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 14:20:50.944053 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.942589 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nxbjn\"" Jul 09 14:20:50.944053 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.942829 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 14:20:50.944053 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.943165 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 14:20:50.944276 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.944147 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 14:20:50.944276 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.944190 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 14:20:50.944373 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.944300 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bjpmw" Jul 09 14:20:50.944373 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.944348 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 14:20:50.944373 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.944364 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 14:20:50.944525 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.944366 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 14:20:50.945074 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.945053 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 14:20:50.945074 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.945073 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cncfb\"" Jul 09 14:20:50.946885 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.946868 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 14:20:50.947005 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.946994 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-sx7xw\"" Jul 09 14:20:50.947245 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.947225 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:50.947331 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:50.947307 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:20:50.950375 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.950355 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n9hkg" Jul 09 14:20:50.952842 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.952816 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 14:20:50.953065 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.953035 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-x7dwh\"" Jul 09 14:20:50.953200 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.953176 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 14:20:50.959956 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.959900 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7rztc" Jul 09 14:20:50.972868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:50.972852 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7rztc" Jul 09 14:20:51.019424 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.019406 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 14:20:51.024055 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024036 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-system-cni-dir\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.024127 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024064 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-os-release\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.024127 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024083 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.024127 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024100 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-sysctl-d\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.024127 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024120 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-hostroot\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.024270 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024135 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5e0a3b3-6bbc-4c39-9b01-035342233924-ovn-node-metrics-cert\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.024270 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024173 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b5e0a3b3-6bbc-4c39-9b01-035342233924-ovnkube-script-lib\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.024270 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024227 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-socket-dir\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.024270 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024260 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-var-lib-kubelet\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.024397 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024293 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-multus-socket-dir-parent\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.024397 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024314 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-run-netns\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.024397 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024331 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01f02794-646c-48f8-8fb1-85cd0a5f9882-kubelet-config\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:51.024397 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024348 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-sysconfig\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.024397 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024363 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-multus-cni-dir\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.024397 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024378 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-run-netns\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.024397 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024393 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-var-lib-openvswitch\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.024642 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024408 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b5e0a3b3-6bbc-4c39-9b01-035342233924-ovnkube-config\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.024642 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024424 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l94h\" (UniqueName: \"kubernetes.io/projected/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-kube-api-access-9l94h\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.024642 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024438 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-system-cni-dir\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.024642 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024452 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-var-lib-cni-bin\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.024642 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024468 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-var-lib-cni-multus\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.024642 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024487 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-etc-kubernetes\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.024642 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024507 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.024642 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024524 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gztz\" (UniqueName: \"kubernetes.io/projected/9205fffb-e805-4302-8d1e-535e1874e263-kube-api-access-4gztz\") pod \"node-resolver-n9hkg\" (UID: \"9205fffb-e805-4302-8d1e-535e1874e263\") " pod="openshift-dns/node-resolver-n9hkg" Jul 09 14:20:51.024642 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024539 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7ba55fe-0514-4431-b5a3-f671cd715374-host-slash\") pod \"iptables-alerter-9x98q\" (UID: \"d7ba55fe-0514-4431-b5a3-f671cd715374\") " pod="openshift-network-operator/iptables-alerter-9x98q" Jul 09 14:20:51.024642 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024555 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-run-ovn-kubernetes\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.024642 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024571 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-cni-netd\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.024642 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024595 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-kubernetes\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.024642 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024622 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca5af9b6-b333-4679-b371-31bc2aa35e8a-tmp\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024646 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-cnibin\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024670 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024698 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-run-ovn\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024721 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-node-log\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024747 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024779 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-sys-fs\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024797 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-host\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024825 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpvrl\" (UniqueName: \"kubernetes.io/projected/45141af1-065c-4341-91be-37adc932e731-kube-api-access-hpvrl\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024847 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e64c211-f85a-4e12-8034-67a6c1e1bb88-host\") pod \"node-ca-ffxnv\" (UID: \"1e64c211-f85a-4e12-8034-67a6c1e1bb88\") " pod="openshift-image-registry/node-ca-ffxnv" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024864 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs\") pod \"network-metrics-daemon-vkk5f\" (UID: \"6050faa0-7272-4fc1-8392-6f072ab52c5a\") " pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024897 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b5e0a3b3-6bbc-4c39-9b01-035342233924-env-overrides\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024923 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024957 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9205fffb-e805-4302-8d1e-535e1874e263-tmp-dir\") pod \"node-resolver-n9hkg\" (UID: \"9205fffb-e805-4302-8d1e-535e1874e263\") " pod="openshift-dns/node-resolver-n9hkg" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.024983 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-registration-dir\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025008 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-cnibin\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.025092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025023 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-kubelet\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025054 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-slash\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025079 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzvrk\" (UniqueName: \"kubernetes.io/projected/b5e0a3b3-6bbc-4c39-9b01-035342233924-kube-api-access-tzvrk\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025102 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-etc-selinux\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025118 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-sysctl-conf\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025138 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-run\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025153 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-os-release\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025175 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45141af1-065c-4341-91be-37adc932e731-cni-binary-copy\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025197 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7ba55fe-0514-4431-b5a3-f671cd715374-iptables-alerter-script\") pod \"iptables-alerter-9x98q\" (UID: \"d7ba55fe-0514-4431-b5a3-f671cd715374\") " pod="openshift-network-operator/iptables-alerter-9x98q" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025218 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sglfc\" (UniqueName: \"kubernetes.io/projected/6050faa0-7272-4fc1-8392-6f072ab52c5a-kube-api-access-sglfc\") pod \"network-metrics-daemon-vkk5f\" (UID: \"6050faa0-7272-4fc1-8392-6f072ab52c5a\") " pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025234 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025249 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znwsq\" (UniqueName: \"kubernetes.io/projected/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-kube-api-access-znwsq\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025292 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-run-openvswitch\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025319 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025342 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-run-k8s-cni-cncf-io\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025360 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/45141af1-065c-4341-91be-37adc932e731-multus-daemon-config\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.025605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025388 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-run-systemd\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025403 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-etc-openvswitch\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025421 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01f02794-646c-48f8-8fb1-85cd0a5f9882-dbus\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025444 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d27fcdf2-e427-43f3-a959-a083bfc76009-konnectivity-ca\") pod \"konnectivity-agent-jc75x\" (UID: \"d27fcdf2-e427-43f3-a959-a083bfc76009\") " pod="kube-system/konnectivity-agent-jc75x" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025464 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-modprobe-d\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025485 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-multus-conf-dir\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025502 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knj9p\" (UniqueName: \"kubernetes.io/projected/d7ba55fe-0514-4431-b5a3-f671cd715374-kube-api-access-knj9p\") pod \"iptables-alerter-9x98q\" (UID: \"d7ba55fe-0514-4431-b5a3-f671cd715374\") " pod="openshift-network-operator/iptables-alerter-9x98q" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025545 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9205fffb-e805-4302-8d1e-535e1874e263-hosts-file\") pod \"node-resolver-n9hkg\" (UID: \"9205fffb-e805-4302-8d1e-535e1874e263\") " pod="openshift-dns/node-resolver-n9hkg" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025565 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d27fcdf2-e427-43f3-a959-a083bfc76009-agent-certs\") pod \"konnectivity-agent-jc75x\" (UID: \"d27fcdf2-e427-43f3-a959-a083bfc76009\") " pod="kube-system/konnectivity-agent-jc75x" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025584 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-systemd\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025604 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-log-socket\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025619 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-sys\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025637 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd5km\" (UniqueName: \"kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km\") pod \"network-check-target-kjc84\" (UID: \"e4b7d8e0-2007-4ead-bf26-96b80cbeeabe\") " pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025658 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-run-multus-certs\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025681 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-systemd-units\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025707 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-cni-bin\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.026099 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025728 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-device-dir\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.026605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025743 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-lib-modules\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.026605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025756 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-tuned\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.026605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025773 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-var-lib-kubelet\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.026605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025801 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e64c211-f85a-4e12-8034-67a6c1e1bb88-serviceca\") pod \"node-ca-ffxnv\" (UID: \"1e64c211-f85a-4e12-8034-67a6c1e1bb88\") " pod="openshift-image-registry/node-ca-ffxnv" Jul 09 14:20:51.026605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025821 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clg4b\" (UniqueName: \"kubernetes.io/projected/1e64c211-f85a-4e12-8034-67a6c1e1bb88-kube-api-access-clg4b\") pod \"node-ca-ffxnv\" (UID: \"1e64c211-f85a-4e12-8034-67a6c1e1bb88\") " pod="openshift-image-registry/node-ca-ffxnv" Jul 09 14:20:51.026605 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.025840 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbr2c\" (UniqueName: \"kubernetes.io/projected/ca5af9b6-b333-4679-b371-31bc2aa35e8a-kube-api-access-xbr2c\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.127031 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.126996 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b5e0a3b3-6bbc-4c39-9b01-035342233924-ovnkube-script-lib\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.127031 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127031 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-socket-dir\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.127324 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127055 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-var-lib-kubelet\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.127324 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127077 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-multus-socket-dir-parent\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.127324 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127137 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-multus-socket-dir-parent\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.127324 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127177 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-run-netns\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.127324 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127204 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01f02794-646c-48f8-8fb1-85cd0a5f9882-kubelet-config\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:51.127324 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127214 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-var-lib-kubelet\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.127324 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127229 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-sysconfig\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.127324 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127256 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-multus-cni-dir\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.127324 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127279 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-run-netns\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.127324 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127283 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01f02794-646c-48f8-8fb1-85cd0a5f9882-kubelet-config\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:51.127324 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127228 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-socket-dir\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.127324 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127283 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-run-netns\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.127324 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127329 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-run-netns\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127329 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-sysconfig\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127331 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-var-lib-openvswitch\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127369 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-var-lib-openvswitch\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127382 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b5e0a3b3-6bbc-4c39-9b01-035342233924-ovnkube-config\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127390 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-multus-cni-dir\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127411 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l94h\" (UniqueName: \"kubernetes.io/projected/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-kube-api-access-9l94h\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127436 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-system-cni-dir\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127451 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-var-lib-cni-bin\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127468 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-var-lib-cni-multus\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127484 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-etc-kubernetes\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127509 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127522 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-var-lib-cni-bin\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127529 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gztz\" (UniqueName: \"kubernetes.io/projected/9205fffb-e805-4302-8d1e-535e1874e263-kube-api-access-4gztz\") pod \"node-resolver-n9hkg\" (UID: \"9205fffb-e805-4302-8d1e-535e1874e263\") " pod="openshift-dns/node-resolver-n9hkg" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127546 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7ba55fe-0514-4431-b5a3-f671cd715374-host-slash\") pod \"iptables-alerter-9x98q\" (UID: \"d7ba55fe-0514-4431-b5a3-f671cd715374\") " pod="openshift-network-operator/iptables-alerter-9x98q" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127552 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-system-cni-dir\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127562 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-run-ovn-kubernetes\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.127997 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127571 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-etc-kubernetes\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127598 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-run-ovn-kubernetes\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127598 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-cni-netd\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127617 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-var-lib-cni-multus\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127632 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-kubernetes\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127652 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b5e0a3b3-6bbc-4c39-9b01-035342233924-ovnkube-script-lib\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127659 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca5af9b6-b333-4679-b371-31bc2aa35e8a-tmp\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127658 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-cni-netd\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127687 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-cnibin\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127727 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-cnibin\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127741 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127773 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-kubernetes\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127772 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-run-ovn\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127809 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-node-log\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127817 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-run-ovn\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127832 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127860 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-sys-fs\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.128893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127882 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-host\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127893 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b5e0a3b3-6bbc-4c39-9b01-035342233924-ovnkube-config\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127910 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpvrl\" (UniqueName: \"kubernetes.io/projected/45141af1-065c-4341-91be-37adc932e731-kube-api-access-hpvrl\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127964 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e64c211-f85a-4e12-8034-67a6c1e1bb88-host\") pod \"node-ca-ffxnv\" (UID: \"1e64c211-f85a-4e12-8034-67a6c1e1bb88\") " pod="openshift-image-registry/node-ca-ffxnv" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127995 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs\") pod \"network-metrics-daemon-vkk5f\" (UID: \"6050faa0-7272-4fc1-8392-6f072ab52c5a\") " pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128003 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128009 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128023 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b5e0a3b3-6bbc-4c39-9b01-035342233924-env-overrides\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128052 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128072 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-sys-fs\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128072 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128089 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-host\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128099 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.127966 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7ba55fe-0514-4431-b5a3-f671cd715374-host-slash\") pod \"iptables-alerter-9x98q\" (UID: \"d7ba55fe-0514-4431-b5a3-f671cd715374\") " pod="openshift-network-operator/iptables-alerter-9x98q" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128054 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-node-log\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128079 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9205fffb-e805-4302-8d1e-535e1874e263-tmp-dir\") pod \"node-resolver-n9hkg\" (UID: \"9205fffb-e805-4302-8d1e-535e1874e263\") " pod="openshift-dns/node-resolver-n9hkg" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128145 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e64c211-f85a-4e12-8034-67a6c1e1bb88-host\") pod \"node-ca-ffxnv\" (UID: \"1e64c211-f85a-4e12-8034-67a6c1e1bb88\") " pod="openshift-image-registry/node-ca-ffxnv" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128162 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-registration-dir\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.129780 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.128165 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128180 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-cnibin\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.128186 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128200 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-kubelet\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128233 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-kubelet\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.128249 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs podName:6050faa0-7272-4fc1-8392-6f072ab52c5a nodeName:}" failed. No retries permitted until 2026-07-09 14:20:51.628206359 +0000 UTC m=+1.994300730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs") pod "network-metrics-daemon-vkk5f" (UID: "6050faa0-7272-4fc1-8392-6f072ab52c5a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128263 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-cnibin\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.128307 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret podName:01f02794-646c-48f8-8fb1-85cd0a5f9882 nodeName:}" failed. No retries permitted until 2026-07-09 14:20:51.628290991 +0000 UTC m=+1.994385357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret") pod "global-pull-secret-syncer-8mtbj" (UID: "01f02794-646c-48f8-8fb1-85cd0a5f9882") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128322 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-slash\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128339 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzvrk\" (UniqueName: \"kubernetes.io/projected/b5e0a3b3-6bbc-4c39-9b01-035342233924-kube-api-access-tzvrk\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128362 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-etc-selinux\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128394 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-sysctl-conf\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128401 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-slash\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128421 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-run\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128406 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b5e0a3b3-6bbc-4c39-9b01-035342233924-env-overrides\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128446 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-os-release\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128470 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45141af1-065c-4341-91be-37adc932e731-cni-binary-copy\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.130902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128488 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-etc-selinux\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128514 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9205fffb-e805-4302-8d1e-535e1874e263-tmp-dir\") pod \"node-resolver-n9hkg\" (UID: \"9205fffb-e805-4302-8d1e-535e1874e263\") " pod="openshift-dns/node-resolver-n9hkg" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128494 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7ba55fe-0514-4431-b5a3-f671cd715374-iptables-alerter-script\") pod \"iptables-alerter-9x98q\" (UID: \"d7ba55fe-0514-4431-b5a3-f671cd715374\") " pod="openshift-network-operator/iptables-alerter-9x98q" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128552 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-os-release\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128553 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-sysctl-conf\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128487 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-run\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128579 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sglfc\" (UniqueName: \"kubernetes.io/projected/6050faa0-7272-4fc1-8392-6f072ab52c5a-kube-api-access-sglfc\") pod \"network-metrics-daemon-vkk5f\" (UID: \"6050faa0-7272-4fc1-8392-6f072ab52c5a\") " pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128607 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128613 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-registration-dir\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128635 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znwsq\" (UniqueName: \"kubernetes.io/projected/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-kube-api-access-znwsq\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128664 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-run-openvswitch\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128692 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128718 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-run-k8s-cni-cncf-io\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128741 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/45141af1-065c-4341-91be-37adc932e731-multus-daemon-config\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128782 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-run-systemd\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128812 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-etc-openvswitch\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01f02794-646c-48f8-8fb1-85cd0a5f9882-dbus\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:51.131868 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128867 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d27fcdf2-e427-43f3-a959-a083bfc76009-konnectivity-ca\") pod \"konnectivity-agent-jc75x\" (UID: \"d27fcdf2-e427-43f3-a959-a083bfc76009\") " pod="kube-system/konnectivity-agent-jc75x" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128891 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-modprobe-d\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128917 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-multus-conf-dir\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128960 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knj9p\" (UniqueName: \"kubernetes.io/projected/d7ba55fe-0514-4431-b5a3-f671cd715374-kube-api-access-knj9p\") pod \"iptables-alerter-9x98q\" (UID: \"d7ba55fe-0514-4431-b5a3-f671cd715374\") " pod="openshift-network-operator/iptables-alerter-9x98q" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.128988 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9205fffb-e805-4302-8d1e-535e1874e263-hosts-file\") pod \"node-resolver-n9hkg\" (UID: \"9205fffb-e805-4302-8d1e-535e1874e263\") " pod="openshift-dns/node-resolver-n9hkg" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129015 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d27fcdf2-e427-43f3-a959-a083bfc76009-agent-certs\") pod \"konnectivity-agent-jc75x\" (UID: \"d27fcdf2-e427-43f3-a959-a083bfc76009\") " pod="kube-system/konnectivity-agent-jc75x" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129042 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-systemd\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129066 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-log-socket\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129091 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-sys\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129117 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd5km\" (UniqueName: \"kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km\") pod \"network-check-target-kjc84\" (UID: \"e4b7d8e0-2007-4ead-bf26-96b80cbeeabe\") " pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129142 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-run-multus-certs\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129155 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-run-systemd\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129165 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-systemd-units\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129184 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129195 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-cni-bin\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129202 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-run-openvswitch\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129222 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-device-dir\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.132608 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129251 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-run-k8s-cni-cncf-io\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129254 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-log-socket\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129273 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-etc-openvswitch\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129257 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-lib-modules\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129287 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/45141af1-065c-4341-91be-37adc932e731-multus-daemon-config\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129313 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-tuned\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129334 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-multus-conf-dir\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129339 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-var-lib-kubelet\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129335 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129113 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/45141af1-065c-4341-91be-37adc932e731-cni-binary-copy\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129364 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e64c211-f85a-4e12-8034-67a6c1e1bb88-serviceca\") pod \"node-ca-ffxnv\" (UID: \"1e64c211-f85a-4e12-8034-67a6c1e1bb88\") " pod="openshift-image-registry/node-ca-ffxnv" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129376 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-lib-modules\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129390 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clg4b\" (UniqueName: \"kubernetes.io/projected/1e64c211-f85a-4e12-8034-67a6c1e1bb88-kube-api-access-clg4b\") pod \"node-ca-ffxnv\" (UID: \"1e64c211-f85a-4e12-8034-67a6c1e1bb88\") " pod="openshift-image-registry/node-ca-ffxnv" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129139 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d7ba55fe-0514-4431-b5a3-f671cd715374-iptables-alerter-script\") pod \"iptables-alerter-9x98q\" (UID: \"d7ba55fe-0514-4431-b5a3-f671cd715374\") " pod="openshift-network-operator/iptables-alerter-9x98q" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129457 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-modprobe-d\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129469 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-run-multus-certs\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129480 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9205fffb-e805-4302-8d1e-535e1874e263-hosts-file\") pod \"node-resolver-n9hkg\" (UID: \"9205fffb-e805-4302-8d1e-535e1874e263\") " pod="openshift-dns/node-resolver-n9hkg" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129562 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-device-dir\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.133192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129535 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01f02794-646c-48f8-8fb1-85cd0a5f9882-dbus\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129602 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-systemd-units\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129622 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5e0a3b3-6bbc-4c39-9b01-035342233924-host-cni-bin\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129644 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbr2c\" (UniqueName: \"kubernetes.io/projected/ca5af9b6-b333-4679-b371-31bc2aa35e8a-kube-api-access-xbr2c\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129658 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d27fcdf2-e427-43f3-a959-a083bfc76009-konnectivity-ca\") pod \"konnectivity-agent-jc75x\" (UID: \"d27fcdf2-e427-43f3-a959-a083bfc76009\") " pod="kube-system/konnectivity-agent-jc75x" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129702 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-sys\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129716 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-system-cni-dir\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129744 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-os-release\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129772 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129820 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-systemd\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129832 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-system-cni-dir\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129886 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-os-release\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.129856 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-host-var-lib-kubelet\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.130095 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-sysctl-d\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.130115 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e64c211-f85a-4e12-8034-67a6c1e1bb88-serviceca\") pod \"node-ca-ffxnv\" (UID: \"1e64c211-f85a-4e12-8034-67a6c1e1bb88\") " pod="openshift-image-registry/node-ca-ffxnv" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.130130 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-hostroot\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.130158 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5e0a3b3-6bbc-4c39-9b01-035342233924-ovn-node-metrics-cert\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.130197 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-sysctl-d\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.133764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.130232 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/45141af1-065c-4341-91be-37adc932e731-hostroot\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.134316 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.130379 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.134316 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.131726 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ca5af9b6-b333-4679-b371-31bc2aa35e8a-etc-tuned\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.134316 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.131773 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca5af9b6-b333-4679-b371-31bc2aa35e8a-tmp\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.134316 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.132681 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d27fcdf2-e427-43f3-a959-a083bfc76009-agent-certs\") pod \"konnectivity-agent-jc75x\" (UID: \"d27fcdf2-e427-43f3-a959-a083bfc76009\") " pod="kube-system/konnectivity-agent-jc75x" Jul 09 14:20:51.134316 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.132826 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b5e0a3b3-6bbc-4c39-9b01-035342233924-ovn-node-metrics-cert\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.146612 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.146577 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l94h\" (UniqueName: \"kubernetes.io/projected/3997ea26-5ed0-4cfd-8e4a-55a94971dbeb-kube-api-access-9l94h\") pod \"aws-ebs-csi-driver-node-vdcss\" (UID: \"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.146726 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.146633 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbr2c\" (UniqueName: \"kubernetes.io/projected/ca5af9b6-b333-4679-b371-31bc2aa35e8a-kube-api-access-xbr2c\") pod \"tuned-m8tlt\" (UID: \"ca5af9b6-b333-4679-b371-31bc2aa35e8a\") " pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.147300 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.147277 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znwsq\" (UniqueName: \"kubernetes.io/projected/2a720bc8-0c99-4249-801c-a6d8e49ff8f3-kube-api-access-znwsq\") pod \"multus-additional-cni-plugins-dz8cc\" (UID: \"2a720bc8-0c99-4249-801c-a6d8e49ff8f3\") " pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.147652 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.147633 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:20:51.148162 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.147657 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:20:51.148162 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.147671 2582 projected.go:196] Error preparing data for projected volume kube-api-access-xd5km for pod openshift-network-diagnostics/network-check-target-kjc84: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:20:51.148162 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.147731 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km podName:e4b7d8e0-2007-4ead-bf26-96b80cbeeabe nodeName:}" failed. No retries permitted until 2026-07-09 14:20:51.647713421 +0000 UTC m=+2.013807789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xd5km" (UniqueName: "kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km") pod "network-check-target-kjc84" (UID: "e4b7d8e0-2007-4ead-bf26-96b80cbeeabe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:20:51.150218 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.150193 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gztz\" (UniqueName: \"kubernetes.io/projected/9205fffb-e805-4302-8d1e-535e1874e263-kube-api-access-4gztz\") pod \"node-resolver-n9hkg\" (UID: \"9205fffb-e805-4302-8d1e-535e1874e263\") " pod="openshift-dns/node-resolver-n9hkg" Jul 09 14:20:51.150318 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.150280 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knj9p\" (UniqueName: \"kubernetes.io/projected/d7ba55fe-0514-4431-b5a3-f671cd715374-kube-api-access-knj9p\") pod \"iptables-alerter-9x98q\" (UID: \"d7ba55fe-0514-4431-b5a3-f671cd715374\") " pod="openshift-network-operator/iptables-alerter-9x98q" Jul 09 14:20:51.150599 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.150577 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpvrl\" (UniqueName: \"kubernetes.io/projected/45141af1-065c-4341-91be-37adc932e731-kube-api-access-hpvrl\") pod \"multus-bjpmw\" (UID: \"45141af1-065c-4341-91be-37adc932e731\") " pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.151445 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.151425 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzvrk\" (UniqueName: \"kubernetes.io/projected/b5e0a3b3-6bbc-4c39-9b01-035342233924-kube-api-access-tzvrk\") pod \"ovnkube-node-l5qkd\" (UID: \"b5e0a3b3-6bbc-4c39-9b01-035342233924\") " pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.151616 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.151600 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clg4b\" (UniqueName: \"kubernetes.io/projected/1e64c211-f85a-4e12-8034-67a6c1e1bb88-kube-api-access-clg4b\") pod \"node-ca-ffxnv\" (UID: \"1e64c211-f85a-4e12-8034-67a6c1e1bb88\") " pod="openshift-image-registry/node-ca-ffxnv" Jul 09 14:20:51.151872 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.151857 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sglfc\" (UniqueName: \"kubernetes.io/projected/6050faa0-7272-4fc1-8392-6f072ab52c5a-kube-api-access-sglfc\") pod \"network-metrics-daemon-vkk5f\" (UID: \"6050faa0-7272-4fc1-8392-6f072ab52c5a\") " pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:20:51.166215 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:51.166180 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a427c6d8749af06dbad4543acc38acf.slice/crio-f4cf4bbbd9a6873f887a305fc8e771a29d154a40987a25c26202623fbd2ed601 WatchSource:0}: Error finding container f4cf4bbbd9a6873f887a305fc8e771a29d154a40987a25c26202623fbd2ed601: Status 404 returned error can't find the container with id f4cf4bbbd9a6873f887a305fc8e771a29d154a40987a25c26202623fbd2ed601 Jul 09 14:20:51.170975 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.170959 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 14:20:51.251990 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.251873 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9x98q" Jul 09 14:20:51.258162 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.258131 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:20:51.262309 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:51.262284 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ba55fe_0514_4431_b5a3_f671cd715374.slice/crio-7d0300297a03dc10c17b804b73a15da956b99af60037552b6ee320a081019ebf WatchSource:0}: Error finding container 7d0300297a03dc10c17b804b73a15da956b99af60037552b6ee320a081019ebf: Status 404 returned error can't find the container with id 7d0300297a03dc10c17b804b73a15da956b99af60037552b6ee320a081019ebf Jul 09 14:20:51.267606 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.267587 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ffxnv" Jul 09 14:20:51.276910 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:51.276890 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e64c211_f85a_4e12_8034_67a6c1e1bb88.slice/crio-279c692de2d97a44d706b5a978d72bd162ef74da5dcac01769a1ec2ebf990be3 WatchSource:0}: Error finding container 279c692de2d97a44d706b5a978d72bd162ef74da5dcac01769a1ec2ebf990be3: Status 404 returned error can't find the container with id 279c692de2d97a44d706b5a978d72bd162ef74da5dcac01769a1ec2ebf990be3 Jul 09 14:20:51.287465 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.287444 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jc75x" Jul 09 14:20:51.296264 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:51.296246 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd27fcdf2_e427_43f3_a959_a083bfc76009.slice/crio-d9d03d46a8ee5afd30aeb05aaba3090b5158007fda817d5988f6d5a0fc134327 WatchSource:0}: Error finding container d9d03d46a8ee5afd30aeb05aaba3090b5158007fda817d5988f6d5a0fc134327: Status 404 returned error can't find the container with id d9d03d46a8ee5afd30aeb05aaba3090b5158007fda817d5988f6d5a0fc134327 Jul 09 14:20:51.302113 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.302097 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" Jul 09 14:20:51.308209 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.308189 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" Jul 09 14:20:51.311058 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:51.310906 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3997ea26_5ed0_4cfd_8e4a_55a94971dbeb.slice/crio-946f67253d619fa61d345025552065033d895518a96bf034a4de56caf53ff83c WatchSource:0}: Error finding container 946f67253d619fa61d345025552065033d895518a96bf034a4de56caf53ff83c: Status 404 returned error can't find the container with id 946f67253d619fa61d345025552065033d895518a96bf034a4de56caf53ff83c Jul 09 14:20:51.311526 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.311510 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:20:51.319248 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:51.319226 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca5af9b6_b333_4679_b371_31bc2aa35e8a.slice/crio-1752a724c437d7b26f35d9da3bd36fbab704693f8ac207362633a423d3b8fd38 WatchSource:0}: Error finding container 1752a724c437d7b26f35d9da3bd36fbab704693f8ac207362633a423d3b8fd38: Status 404 returned error can't find the container with id 1752a724c437d7b26f35d9da3bd36fbab704693f8ac207362633a423d3b8fd38 Jul 09 14:20:51.328070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.328054 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dz8cc" Jul 09 14:20:51.337677 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:51.337651 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a720bc8_0c99_4249_801c_a6d8e49ff8f3.slice/crio-acb7c8687ae5ab566ff207e9aa4e82ed381c612105cec709030128a1335f60a7 WatchSource:0}: Error finding container acb7c8687ae5ab566ff207e9aa4e82ed381c612105cec709030128a1335f60a7: Status 404 returned error can't find the container with id acb7c8687ae5ab566ff207e9aa4e82ed381c612105cec709030128a1335f60a7 Jul 09 14:20:51.344422 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.344403 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:20:51.353127 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:51.353106 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5e0a3b3_6bbc_4c39_9b01_035342233924.slice/crio-e4429283a6d66b23234ea06e851aebb41593eb1f776d4d5c62545617fc341cad WatchSource:0}: Error finding container e4429283a6d66b23234ea06e851aebb41593eb1f776d4d5c62545617fc341cad: Status 404 returned error can't find the container with id e4429283a6d66b23234ea06e851aebb41593eb1f776d4d5c62545617fc341cad Jul 09 14:20:51.360295 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.360277 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bjpmw" Jul 09 14:20:51.365244 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.365227 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n9hkg" Jul 09 14:20:51.368656 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:51.368638 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45141af1_065c_4341_91be_37adc932e731.slice/crio-f69f2b9b910798a7614c0d3cfab3deed8cf7c62f44024808a7e8a208645b1379 WatchSource:0}: Error finding container f69f2b9b910798a7614c0d3cfab3deed8cf7c62f44024808a7e8a208645b1379: Status 404 returned error can't find the container with id f69f2b9b910798a7614c0d3cfab3deed8cf7c62f44024808a7e8a208645b1379 Jul 09 14:20:51.373993 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:20:51.373972 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9205fffb_e805_4302_8d1e_535e1874e263.slice/crio-ac772f9c2f6f5184d6e5f27e27341da2641c9b7548fc887962780afc74874b1a WatchSource:0}: Error finding container ac772f9c2f6f5184d6e5f27e27341da2641c9b7548fc887962780afc74874b1a: Status 404 returned error can't find the container with id ac772f9c2f6f5184d6e5f27e27341da2641c9b7548fc887962780afc74874b1a Jul 09 14:20:51.633022 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.632932 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs\") pod \"network-metrics-daemon-vkk5f\" (UID: \"6050faa0-7272-4fc1-8392-6f072ab52c5a\") " pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:20:51.633022 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.632981 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:51.633234 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.633040 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:20:51.633234 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.633079 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:20:51.633234 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.633120 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs podName:6050faa0-7272-4fc1-8392-6f072ab52c5a nodeName:}" failed. No retries permitted until 2026-07-09 14:20:52.633096163 +0000 UTC m=+2.999190529 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs") pod "network-metrics-daemon-vkk5f" (UID: "6050faa0-7272-4fc1-8392-6f072ab52c5a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:20:51.633234 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.633137 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret podName:01f02794-646c-48f8-8fb1-85cd0a5f9882 nodeName:}" failed. No retries permitted until 2026-07-09 14:20:52.633129775 +0000 UTC m=+2.999224140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret") pod "global-pull-secret-syncer-8mtbj" (UID: "01f02794-646c-48f8-8fb1-85cd0a5f9882") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:20:51.733573 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.733530 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd5km\" (UniqueName: \"kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km\") pod \"network-check-target-kjc84\" (UID: \"e4b7d8e0-2007-4ead-bf26-96b80cbeeabe\") " pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:20:51.733775 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.733709 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:20:51.733775 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.733732 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:20:51.733775 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.733746 2582 projected.go:196] Error preparing data for projected volume kube-api-access-xd5km for pod openshift-network-diagnostics/network-check-target-kjc84: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:20:51.733993 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:51.733809 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km podName:e4b7d8e0-2007-4ead-bf26-96b80cbeeabe nodeName:}" failed. No retries permitted until 2026-07-09 14:20:52.733788603 +0000 UTC m=+3.099882990 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xd5km" (UniqueName: "kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km") pod "network-check-target-kjc84" (UID: "e4b7d8e0-2007-4ead-bf26-96b80cbeeabe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:20:51.902557 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.902478 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:20:51.976103 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.976030 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 14:15:51 +0000 UTC" deadline="2028-04-18 22:04:34.79733564 +0000 UTC" Jul 09 14:20:51.976103 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:51.976100 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15583h43m42.821239854s" Jul 09 14:20:52.043347 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.043192 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" event={"ID":"b5e0a3b3-6bbc-4c39-9b01-035342233924","Type":"ContainerStarted","Data":"e4429283a6d66b23234ea06e851aebb41593eb1f776d4d5c62545617fc341cad"} Jul 09 14:20:52.048146 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.048111 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8cc" event={"ID":"2a720bc8-0c99-4249-801c-a6d8e49ff8f3","Type":"ContainerStarted","Data":"acb7c8687ae5ab566ff207e9aa4e82ed381c612105cec709030128a1335f60a7"} Jul 09 14:20:52.056572 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.056539 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" event={"ID":"ca5af9b6-b333-4679-b371-31bc2aa35e8a","Type":"ContainerStarted","Data":"1752a724c437d7b26f35d9da3bd36fbab704693f8ac207362633a423d3b8fd38"} Jul 09 14:20:52.069930 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.069895 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" event={"ID":"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb","Type":"ContainerStarted","Data":"946f67253d619fa61d345025552065033d895518a96bf034a4de56caf53ff83c"} Jul 09 14:20:52.076625 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.076569 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jc75x" event={"ID":"d27fcdf2-e427-43f3-a959-a083bfc76009","Type":"ContainerStarted","Data":"d9d03d46a8ee5afd30aeb05aaba3090b5158007fda817d5988f6d5a0fc134327"} Jul 09 14:20:52.080281 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.080223 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ffxnv" event={"ID":"1e64c211-f85a-4e12-8034-67a6c1e1bb88","Type":"ContainerStarted","Data":"279c692de2d97a44d706b5a978d72bd162ef74da5dcac01769a1ec2ebf990be3"} Jul 09 14:20:52.083076 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.083049 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9x98q" event={"ID":"d7ba55fe-0514-4431-b5a3-f671cd715374","Type":"ContainerStarted","Data":"7d0300297a03dc10c17b804b73a15da956b99af60037552b6ee320a081019ebf"} Jul 09 14:20:52.086193 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.086166 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal" event={"ID":"0a427c6d8749af06dbad4543acc38acf","Type":"ContainerStarted","Data":"f4cf4bbbd9a6873f887a305fc8e771a29d154a40987a25c26202623fbd2ed601"} Jul 09 14:20:52.088091 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.088046 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n9hkg" event={"ID":"9205fffb-e805-4302-8d1e-535e1874e263","Type":"ContainerStarted","Data":"ac772f9c2f6f5184d6e5f27e27341da2641c9b7548fc887962780afc74874b1a"} Jul 09 14:20:52.092434 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.092407 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bjpmw" event={"ID":"45141af1-065c-4341-91be-37adc932e731","Type":"ContainerStarted","Data":"f69f2b9b910798a7614c0d3cfab3deed8cf7c62f44024808a7e8a208645b1379"} Jul 09 14:20:52.640332 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.640223 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs\") pod \"network-metrics-daemon-vkk5f\" (UID: \"6050faa0-7272-4fc1-8392-6f072ab52c5a\") " pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:20:52.640332 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.640282 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:52.640577 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:52.640443 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:20:52.640577 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:52.640503 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret podName:01f02794-646c-48f8-8fb1-85cd0a5f9882 nodeName:}" failed. No retries permitted until 2026-07-09 14:20:54.640487749 +0000 UTC m=+5.006582115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret") pod "global-pull-secret-syncer-8mtbj" (UID: "01f02794-646c-48f8-8fb1-85cd0a5f9882") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:20:52.640870 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:52.640850 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:20:52.640974 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:52.640909 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs podName:6050faa0-7272-4fc1-8392-6f072ab52c5a nodeName:}" failed. No retries permitted until 2026-07-09 14:20:54.640890789 +0000 UTC m=+5.006985159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs") pod "network-metrics-daemon-vkk5f" (UID: "6050faa0-7272-4fc1-8392-6f072ab52c5a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:20:52.741413 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.741372 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd5km\" (UniqueName: \"kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km\") pod \"network-check-target-kjc84\" (UID: \"e4b7d8e0-2007-4ead-bf26-96b80cbeeabe\") " pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:20:52.741628 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:52.741541 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:20:52.741628 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:52.741563 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:20:52.741628 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:52.741577 2582 projected.go:196] Error preparing data for projected volume kube-api-access-xd5km for pod openshift-network-diagnostics/network-check-target-kjc84: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:20:52.741806 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:52.741641 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km podName:e4b7d8e0-2007-4ead-bf26-96b80cbeeabe nodeName:}" failed. No retries permitted until 2026-07-09 14:20:54.741620523 +0000 UTC m=+5.107714891 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xd5km" (UniqueName: "kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km") pod "network-check-target-kjc84" (UID: "e4b7d8e0-2007-4ead-bf26-96b80cbeeabe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:20:52.977014 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.976904 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 14:15:51 +0000 UTC" deadline="2028-04-25 10:24:42.145735386 +0000 UTC" Jul 09 14:20:52.977014 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:52.976946 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15740h3m49.168795176s" Jul 09 14:20:53.031573 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:53.031539 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:20:53.031746 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:53.031678 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:20:53.032176 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:53.032152 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:20:53.032305 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:53.032282 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:20:53.032374 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:53.032359 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:53.032472 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:53.032448 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:20:53.096985 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:53.096961 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 14:20:53.116097 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:53.115928 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-93.ec2.internal" event={"ID":"0b2614806c45fdfd93e51f60f22bdd4c","Type":"ContainerStarted","Data":"b2f6a658551d1376a4cf8862d8fa4dce6c3ed2b9a5bb69b080b8837a68cc60c4"} Jul 09 14:20:54.659050 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:54.658220 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs\") pod \"network-metrics-daemon-vkk5f\" (UID: \"6050faa0-7272-4fc1-8392-6f072ab52c5a\") " pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:20:54.659050 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:54.658269 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:54.659050 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:54.658437 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:20:54.659050 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:54.658497 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret podName:01f02794-646c-48f8-8fb1-85cd0a5f9882 nodeName:}" failed. No retries permitted until 2026-07-09 14:20:58.658478327 +0000 UTC m=+9.024572708 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret") pod "global-pull-secret-syncer-8mtbj" (UID: "01f02794-646c-48f8-8fb1-85cd0a5f9882") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:20:54.659050 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:54.658939 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:20:54.659050 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:54.659020 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs podName:6050faa0-7272-4fc1-8392-6f072ab52c5a nodeName:}" failed. No retries permitted until 2026-07-09 14:20:58.659001074 +0000 UTC m=+9.025095445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs") pod "network-metrics-daemon-vkk5f" (UID: "6050faa0-7272-4fc1-8392-6f072ab52c5a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:20:54.763667 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:54.763598 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd5km\" (UniqueName: \"kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km\") pod \"network-check-target-kjc84\" (UID: \"e4b7d8e0-2007-4ead-bf26-96b80cbeeabe\") " pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:20:54.763835 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:54.763747 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:20:54.763835 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:54.763768 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:20:54.763835 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:54.763778 2582 projected.go:196] Error preparing data for projected volume kube-api-access-xd5km for pod openshift-network-diagnostics/network-check-target-kjc84: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:20:54.763983 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:54.763836 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km podName:e4b7d8e0-2007-4ead-bf26-96b80cbeeabe nodeName:}" failed. No retries permitted until 2026-07-09 14:20:58.763820019 +0000 UTC m=+9.129914385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xd5km" (UniqueName: "kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km") pod "network-check-target-kjc84" (UID: "e4b7d8e0-2007-4ead-bf26-96b80cbeeabe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:20:55.031853 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:55.031767 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:20:55.032075 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:55.031902 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:20:55.032075 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:55.031989 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:55.032211 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:55.032077 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:20:55.032211 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:55.032131 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:20:55.032311 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:55.032206 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:20:57.031975 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:57.031936 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:57.032482 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:57.032080 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:20:57.032482 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:57.032085 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:20:57.032482 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:57.032161 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:20:57.032482 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:57.032238 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:20:57.032482 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:57.032376 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:20:58.698312 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:58.698223 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs\") pod \"network-metrics-daemon-vkk5f\" (UID: \"6050faa0-7272-4fc1-8392-6f072ab52c5a\") " pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:20:58.698312 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:58.698288 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:58.699197 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:58.698479 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:20:58.699197 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:58.698549 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret podName:01f02794-646c-48f8-8fb1-85cd0a5f9882 nodeName:}" failed. No retries permitted until 2026-07-09 14:21:06.698529611 +0000 UTC m=+17.064623978 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret") pod "global-pull-secret-syncer-8mtbj" (UID: "01f02794-646c-48f8-8fb1-85cd0a5f9882") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:20:58.699197 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:58.698925 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:20:58.699197 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:58.699065 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs podName:6050faa0-7272-4fc1-8392-6f072ab52c5a nodeName:}" failed. No retries permitted until 2026-07-09 14:21:06.699043622 +0000 UTC m=+17.065138003 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs") pod "network-metrics-daemon-vkk5f" (UID: "6050faa0-7272-4fc1-8392-6f072ab52c5a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:20:58.799304 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:58.799259 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd5km\" (UniqueName: \"kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km\") pod \"network-check-target-kjc84\" (UID: \"e4b7d8e0-2007-4ead-bf26-96b80cbeeabe\") " pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:20:58.799519 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:58.799500 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:20:58.799580 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:58.799527 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:20:58.799580 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:58.799544 2582 projected.go:196] Error preparing data for projected volume kube-api-access-xd5km for pod openshift-network-diagnostics/network-check-target-kjc84: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:20:58.799683 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:58.799610 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km podName:e4b7d8e0-2007-4ead-bf26-96b80cbeeabe nodeName:}" failed. No retries permitted until 2026-07-09 14:21:06.799589159 +0000 UTC m=+17.165683530 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xd5km" (UniqueName: "kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km") pod "network-check-target-kjc84" (UID: "e4b7d8e0-2007-4ead-bf26-96b80cbeeabe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:20:59.032373 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:59.032093 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:20:59.032559 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:59.032105 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:20:59.032559 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:59.032438 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:20:59.032559 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:20:59.032105 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:20:59.032559 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:59.032522 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:20:59.032724 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:20:59.032597 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:21:01.032055 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:01.032024 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:01.032609 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:01.032146 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:21:01.032609 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:01.032038 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:01.032609 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:01.032239 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:21:01.032609 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:01.032025 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:01.032609 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:01.032311 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:21:03.031761 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:03.031687 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:03.031761 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:03.031705 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:03.031761 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:03.031711 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:03.032370 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:03.031815 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:21:03.032370 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:03.031982 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:21:03.032370 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:03.032088 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:21:05.031663 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:05.031631 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:05.032121 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:05.031633 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:05.032121 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:05.031741 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:21:05.032121 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:05.031632 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:05.032121 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:05.031811 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:21:05.032121 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:05.031889 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:21:06.760977 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:06.760933 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs\") pod \"network-metrics-daemon-vkk5f\" (UID: \"6050faa0-7272-4fc1-8392-6f072ab52c5a\") " pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:06.760977 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:06.760986 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:06.761497 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:06.761104 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:21:06.761497 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:06.761108 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:21:06.761497 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:06.761174 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs podName:6050faa0-7272-4fc1-8392-6f072ab52c5a nodeName:}" failed. No retries permitted until 2026-07-09 14:21:22.761152143 +0000 UTC m=+33.127246527 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs") pod "network-metrics-daemon-vkk5f" (UID: "6050faa0-7272-4fc1-8392-6f072ab52c5a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:21:06.761497 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:06.761200 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret podName:01f02794-646c-48f8-8fb1-85cd0a5f9882 nodeName:}" failed. No retries permitted until 2026-07-09 14:21:22.761189641 +0000 UTC m=+33.127284012 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret") pod "global-pull-secret-syncer-8mtbj" (UID: "01f02794-646c-48f8-8fb1-85cd0a5f9882") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:21:06.862148 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:06.862097 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd5km\" (UniqueName: \"kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km\") pod \"network-check-target-kjc84\" (UID: \"e4b7d8e0-2007-4ead-bf26-96b80cbeeabe\") " pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:06.862336 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:06.862291 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:21:06.862336 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:06.862322 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:21:06.862336 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:06.862335 2582 projected.go:196] Error preparing data for projected volume kube-api-access-xd5km for pod openshift-network-diagnostics/network-check-target-kjc84: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:21:06.862491 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:06.862404 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km podName:e4b7d8e0-2007-4ead-bf26-96b80cbeeabe nodeName:}" failed. No retries permitted until 2026-07-09 14:21:22.862383318 +0000 UTC m=+33.228477699 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xd5km" (UniqueName: "kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km") pod "network-check-target-kjc84" (UID: "e4b7d8e0-2007-4ead-bf26-96b80cbeeabe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:21:07.032156 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:07.032080 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:07.032318 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:07.032186 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:21:07.032318 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:07.032080 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:07.032318 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:07.032214 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:07.032464 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:07.032336 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:21:07.032464 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:07.032433 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:21:09.032163 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:09.032133 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:09.032666 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:09.032132 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:09.032666 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:09.032255 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:21:09.032666 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:09.032133 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:09.032666 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:09.032334 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:21:09.032666 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:09.032392 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:21:11.031645 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:11.031605 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:11.032133 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:11.031606 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:11.032133 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:11.031751 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:21:11.032133 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:11.031768 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:11.032133 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:11.031831 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:21:11.032133 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:11.031923 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:21:12.156908 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:12.156648 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bjpmw" event={"ID":"45141af1-065c-4341-91be-37adc932e731","Type":"ContainerStarted","Data":"3c7f6f17a2a5c06578cdc5101374df5a693a042ac838676478fddfba9c925a2f"} Jul 09 14:21:12.159530 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:12.159167 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" event={"ID":"b5e0a3b3-6bbc-4c39-9b01-035342233924","Type":"ContainerStarted","Data":"23b39d9d69cfe38172dd68d6fd88fac085d7c2a61b8b9c1ee8dfe4cf1f01b445"} Jul 09 14:21:12.159530 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:12.159313 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" event={"ID":"b5e0a3b3-6bbc-4c39-9b01-035342233924","Type":"ContainerStarted","Data":"2646d39a7cd69edefce3edaf5a91092e70bd486607c60afe8dc39eac78f3cadc"} Jul 09 14:21:12.160611 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:12.160590 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" event={"ID":"ca5af9b6-b333-4679-b371-31bc2aa35e8a","Type":"ContainerStarted","Data":"fd69f76e90ce88fe0fbb72d50461ca14456e83ea53cd03c86907610793f2a060"} Jul 09 14:21:12.161717 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:12.161701 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-93.ec2.internal" event={"ID":"0b2614806c45fdfd93e51f60f22bdd4c","Type":"ContainerStarted","Data":"ed642ad4752a2942b3ecff5c5fd82b420d00af6b0cfa753307060b41261cf381"} Jul 09 14:21:12.195128 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:12.195086 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bjpmw" podStartSLOduration=1.6285694130000001 podStartE2EDuration="22.195070829s" totalImagesPullingTime="20.566501416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:12.194598906 +0000 UTC m=+22.560693293" watchObservedRunningTime="2026-07-09 14:21:12.195070829 +0000 UTC m=+22.561165214" Jul 09 14:21:12.227076 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:12.227036 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m8tlt" podStartSLOduration=1.673188289 podStartE2EDuration="22.227019933s" totalImagesPullingTime="20.553831644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:12.226957265 +0000 UTC m=+22.593051652" watchObservedRunningTime="2026-07-09 14:21:12.227019933 +0000 UTC m=+22.593114319" Jul 09 14:21:12.227798 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:12.227566 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-93.ec2.internal" podStartSLOduration=21.227556716 podStartE2EDuration="21.227556716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:12.209388418 +0000 UTC m=+22.575482804" watchObservedRunningTime="2026-07-09 14:21:12.227556716 +0000 UTC m=+22.593651103" Jul 09 14:21:13.031518 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.031350 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:13.031674 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.031353 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:13.031674 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:13.031586 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:21:13.031674 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:13.031666 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:21:13.031768 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.031382 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:13.031768 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:13.031744 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:21:13.165820 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.165783 2582 generic.go:356] "Generic (PLEG): container finished" podID="0a427c6d8749af06dbad4543acc38acf" containerID="91a90b4311c433ffa4b5390f582b803a8c8be05c7d227cab68b25358029026e9" exitCode=0 Jul 09 14:21:13.166293 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.165887 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal" event={"ID":"0a427c6d8749af06dbad4543acc38acf","Type":"ContainerDied","Data":"91a90b4311c433ffa4b5390f582b803a8c8be05c7d227cab68b25358029026e9"} Jul 09 14:21:13.167497 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.167464 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n9hkg" event={"ID":"9205fffb-e805-4302-8d1e-535e1874e263","Type":"ContainerStarted","Data":"13d80a18cec4f94ed4df1afd96b3f035650a206e4d3494318e00cad5975df022"} Jul 09 14:21:13.170672 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.170645 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" event={"ID":"b5e0a3b3-6bbc-4c39-9b01-035342233924","Type":"ContainerStarted","Data":"e6ed70fe24dc0731beca8d39a7aa034235e268cd8efb7e5a7e332db8e1956486"} Jul 09 14:21:13.170786 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.170723 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" event={"ID":"b5e0a3b3-6bbc-4c39-9b01-035342233924","Type":"ContainerStarted","Data":"50ebad51ee7506c885c1649650700f12bdb39f33cb49c3965af3acc0770fdb1e"} Jul 09 14:21:13.170786 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.170763 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" event={"ID":"b5e0a3b3-6bbc-4c39-9b01-035342233924","Type":"ContainerStarted","Data":"7d5bae7e1792543487d772e2e07430faca6fe6489d2b9bbdfb9fb95c2aaac018"} Jul 09 14:21:13.170913 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.170795 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" event={"ID":"b5e0a3b3-6bbc-4c39-9b01-035342233924","Type":"ContainerStarted","Data":"297546b7e55db202ef583968f1497738ec24652dd763f45cdb16e3f89fd49a4a"} Jul 09 14:21:13.172336 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.172312 2582 generic.go:356] "Generic (PLEG): container finished" podID="2a720bc8-0c99-4249-801c-a6d8e49ff8f3" containerID="36d9ffe196aef22d3a9373c8e3cb409fab05ff39e3dc2332674b8066e5a75879" exitCode=0 Jul 09 14:21:13.172503 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.172386 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8cc" event={"ID":"2a720bc8-0c99-4249-801c-a6d8e49ff8f3","Type":"ContainerDied","Data":"36d9ffe196aef22d3a9373c8e3cb409fab05ff39e3dc2332674b8066e5a75879"} Jul 09 14:21:13.173961 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.173859 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" event={"ID":"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb","Type":"ContainerStarted","Data":"bc934839b81e60f29edddf453bafae992dc9ca557f7c2a0dfaa9c4ae95fd7b25"} Jul 09 14:21:13.177821 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.177799 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jc75x" event={"ID":"d27fcdf2-e427-43f3-a959-a083bfc76009","Type":"ContainerStarted","Data":"fe4d05dfb9832bf507e8a8a08b346db09fb2e6cd69d877ea99679250d3b20b36"} Jul 09 14:21:13.179195 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.179165 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ffxnv" event={"ID":"1e64c211-f85a-4e12-8034-67a6c1e1bb88","Type":"ContainerStarted","Data":"98a38c9525360bb5d7135ced93299eda9708ea6e8bdcf0a4bc625cbed28be24f"} Jul 09 14:21:13.180813 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.180542 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9x98q" event={"ID":"d7ba55fe-0514-4431-b5a3-f671cd715374","Type":"ContainerStarted","Data":"e09dc817882978a5166c23429b672b619af2665c071effe8c3da2bf0a942cade"} Jul 09 14:21:13.201254 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.201216 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-n9hkg" podStartSLOduration=2.6989665929999997 podStartE2EDuration="23.201202642s" totalImagesPullingTime="20.502236049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:13.200843949 +0000 UTC m=+23.566938335" watchObservedRunningTime="2026-07-09 14:21:13.201202642 +0000 UTC m=+23.567297027" Jul 09 14:21:13.218702 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.218653 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9x98q" podStartSLOduration=10.698039903 podStartE2EDuration="23.218639431s" totalImagesPullingTime="12.520599528s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:13.218132663 +0000 UTC m=+23.584227049" watchObservedRunningTime="2026-07-09 14:21:13.218639431 +0000 UTC m=+23.584733799" Jul 09 14:21:13.235627 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.235573 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ffxnv" podStartSLOduration=10.729469754 podStartE2EDuration="23.235561025s" totalImagesPullingTime="12.506091271s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:13.234961234 +0000 UTC m=+23.601055633" watchObservedRunningTime="2026-07-09 14:21:13.235561025 +0000 UTC m=+23.601655414" Jul 09 14:21:13.263395 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.263348 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jc75x" podStartSLOduration=10.777099795 podStartE2EDuration="23.263333801s" totalImagesPullingTime="12.486234006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:13.263196496 +0000 UTC m=+23.629290882" watchObservedRunningTime="2026-07-09 14:21:13.263333801 +0000 UTC m=+23.629428187" Jul 09 14:21:13.498055 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.498027 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 14:21:13.694180 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.694104 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jc75x" Jul 09 14:21:13.694689 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:13.694672 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jc75x" Jul 09 14:21:14.002188 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:14.001998 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T14:21:13.49804848Z","UUID":"7c578069-d684-43dd-aeea-d4b37292a9b6","Handler":null,"Name":"","Endpoint":""} Jul 09 14:21:14.003407 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:14.003389 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 14:21:14.003537 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:14.003423 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 14:21:14.184742 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:14.184708 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal" event={"ID":"0a427c6d8749af06dbad4543acc38acf","Type":"ContainerStarted","Data":"fd945381a499f279ced1d2ee7d19748ecc99c755b72414cfc21aa9393d8c3151"} Jul 09 14:21:14.186346 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:14.186321 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" event={"ID":"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb","Type":"ContainerStarted","Data":"447a8b97c4526e35829e860453fa473442754b111f5ce29eebfa85f97ead3bcf"} Jul 09 14:21:14.186793 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:14.186770 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jc75x" Jul 09 14:21:14.187088 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:14.187071 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jc75x" Jul 09 14:21:14.216204 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:14.216155 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-93.ec2.internal" podStartSLOduration=23.216126235 podStartE2EDuration="23.216126235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:14.200265503 +0000 UTC m=+24.566359891" watchObservedRunningTime="2026-07-09 14:21:14.216126235 +0000 UTC m=+24.582220623" Jul 09 14:21:14.303455 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:14.303375 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jc75x_d27fcdf2-e427-43f3-a959-a083bfc76009/konnectivity-agent/0.log" limit=1024 Jul 09 14:21:14.400351 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:14.400318 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jc75x_d27fcdf2-e427-43f3-a959-a083bfc76009/konnectivity-agent/0.log" limit=1024 Jul 09 14:21:14.932173 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:14.932148 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jc75x_d27fcdf2-e427-43f3-a959-a083bfc76009/konnectivity-agent/0.log" limit=1024 Jul 09 14:21:15.032237 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:15.032210 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:15.032367 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:15.032246 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:15.032367 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:15.032329 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:21:15.032367 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:15.032219 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:15.032564 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:15.032460 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:21:15.032564 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:15.032522 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:21:15.032648 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:15.032624 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jc75x_d27fcdf2-e427-43f3-a959-a083bfc76009/konnectivity-agent/0.log" limit=1024 Jul 09 14:21:15.193372 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:15.193330 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" event={"ID":"b5e0a3b3-6bbc-4c39-9b01-035342233924","Type":"ContainerStarted","Data":"d36f4e91e78535950cdf4fb1ff55558be428c79e40041d31d7941dbd2074783a"} Jul 09 14:21:15.195485 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:15.195452 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" event={"ID":"3997ea26-5ed0-4cfd-8e4a-55a94971dbeb","Type":"ContainerStarted","Data":"20cfa87ce8d811ee9f19ac67404e0305ee0c77bb3c2cf36cbcdae5fe18caf06b"} Jul 09 14:21:15.215951 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:15.215901 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vdcss" podStartSLOduration=10.506672663 podStartE2EDuration="25.215886325s" totalImagesPullingTime="14.709213662s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:20:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:15.215717752 +0000 UTC m=+25.581812137" watchObservedRunningTime="2026-07-09 14:21:15.215886325 +0000 UTC m=+25.581980701" Jul 09 14:21:16.906723 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:16.906690 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jc75x_d27fcdf2-e427-43f3-a959-a083bfc76009/konnectivity-agent/0.log" limit=1024 Jul 09 14:21:17.009403 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:17.009243 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jc75x_d27fcdf2-e427-43f3-a959-a083bfc76009/konnectivity-agent/0.log" limit=1024 Jul 09 14:21:17.032298 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:17.032274 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:17.032406 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:17.032302 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:17.032406 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:17.032307 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:17.032487 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:17.032423 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:21:17.032536 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:17.032500 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:21:17.032625 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:17.032607 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:21:17.204419 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:17.204328 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" event={"ID":"b5e0a3b3-6bbc-4c39-9b01-035342233924","Type":"ContainerStarted","Data":"d463c9c236b7a23d3f884005e55b6fdb8e80eebe0a15bb08a3fd3da05fc997a5"} Jul 09 14:21:17.204828 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:17.204786 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:21:17.204966 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:17.204897 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:21:17.223178 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:17.223131 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:21:17.223421 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:17.223387 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:21:17.235800 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:17.235751 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" podStartSLOduration=6.655073317 podStartE2EDuration="27.23573641s" totalImagesPullingTime="20.580663093s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:17.235506454 +0000 UTC m=+27.601600857" watchObservedRunningTime="2026-07-09 14:21:17.23573641 +0000 UTC m=+27.601830796" Jul 09 14:21:18.207651 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:18.206776 2582 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 09 14:21:18.834411 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:18.834379 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jc75x_d27fcdf2-e427-43f3-a959-a083bfc76009/konnectivity-agent/0.log" limit=1024 Jul 09 14:21:18.935244 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:18.935084 2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jc75x_d27fcdf2-e427-43f3-a959-a083bfc76009/konnectivity-agent/0.log" limit=1024 Jul 09 14:21:19.031568 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:19.031532 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:19.031568 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:19.031556 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:19.031747 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:19.031643 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:19.031747 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:19.031663 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:21:19.031810 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:19.031744 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:21:19.031810 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:19.031784 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:21:19.100820 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:19.100717 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8mtbj"] Jul 09 14:21:19.104075 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:19.104051 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kjc84"] Jul 09 14:21:19.104657 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:19.104635 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vkk5f"] Jul 09 14:21:19.208803 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:19.208780 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:19.208803 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:19.208802 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:19.209242 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:19.208857 2582 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 09 14:21:19.209242 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:19.208783 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:19.209242 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:19.208901 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:21:19.209242 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:19.209207 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:21:19.209369 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:19.209268 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:21:20.111332 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:20.111300 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:21:20.212609 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:20.212575 2582 generic.go:356] "Generic (PLEG): container finished" podID="2a720bc8-0c99-4249-801c-a6d8e49ff8f3" containerID="a81973d95f788124e1509b3fbd1693e6ab1aa5b4c8251fcf3075fbafa3bd6db7" exitCode=0 Jul 09 14:21:20.213081 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:20.212656 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8cc" event={"ID":"2a720bc8-0c99-4249-801c-a6d8e49ff8f3","Type":"ContainerDied","Data":"a81973d95f788124e1509b3fbd1693e6ab1aa5b4c8251fcf3075fbafa3bd6db7"} Jul 09 14:21:21.032193 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:21.032156 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:21.032193 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:21.032171 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:21.032193 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:21.032185 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:21.032449 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:21.032276 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:21:21.032449 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:21.032420 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:21:21.032529 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:21.032506 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:21:21.217958 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:21.217926 2582 generic.go:356] "Generic (PLEG): container finished" podID="2a720bc8-0c99-4249-801c-a6d8e49ff8f3" containerID="47be7a452fd02b8105e147611f74b835b5de549526d5dac5b604abb51c4fbc52" exitCode=0 Jul 09 14:21:21.218318 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:21.217973 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8cc" event={"ID":"2a720bc8-0c99-4249-801c-a6d8e49ff8f3","Type":"ContainerDied","Data":"47be7a452fd02b8105e147611f74b835b5de549526d5dac5b604abb51c4fbc52"} Jul 09 14:21:22.223395 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:22.223370 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8cc" event={"ID":"2a720bc8-0c99-4249-801c-a6d8e49ff8f3","Type":"ContainerStarted","Data":"4a4ee14e4bedf40369a4ed0f6ef2e6701c39817a9749f90927678de1467289f5"} Jul 09 14:21:22.777735 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:22.777696 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs\") pod \"network-metrics-daemon-vkk5f\" (UID: \"6050faa0-7272-4fc1-8392-6f072ab52c5a\") " pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:22.777735 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:22.777736 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:22.778022 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:22.777844 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:21:22.778022 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:22.777853 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 14:21:22.778022 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:22.777924 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret podName:01f02794-646c-48f8-8fb1-85cd0a5f9882 nodeName:}" failed. No retries permitted until 2026-07-09 14:21:54.777905397 +0000 UTC m=+65.143999775 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret") pod "global-pull-secret-syncer-8mtbj" (UID: "01f02794-646c-48f8-8fb1-85cd0a5f9882") : object "kube-system"/"original-pull-secret" not registered Jul 09 14:21:22.778022 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:22.777942 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs podName:6050faa0-7272-4fc1-8392-6f072ab52c5a nodeName:}" failed. No retries permitted until 2026-07-09 14:21:54.777933642 +0000 UTC m=+65.144028034 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs") pod "network-metrics-daemon-vkk5f" (UID: "6050faa0-7272-4fc1-8392-6f072ab52c5a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 14:21:22.878133 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:22.878100 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd5km\" (UniqueName: \"kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km\") pod \"network-check-target-kjc84\" (UID: \"e4b7d8e0-2007-4ead-bf26-96b80cbeeabe\") " pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:22.878274 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:22.878253 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 14:21:22.878274 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:22.878273 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 14:21:22.878358 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:22.878283 2582 projected.go:196] Error preparing data for projected volume kube-api-access-xd5km for pod openshift-network-diagnostics/network-check-target-kjc84: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:21:22.878358 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:22.878328 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km podName:e4b7d8e0-2007-4ead-bf26-96b80cbeeabe nodeName:}" failed. No retries permitted until 2026-07-09 14:21:54.878315337 +0000 UTC m=+65.244409701 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xd5km" (UniqueName: "kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km") pod "network-check-target-kjc84" (UID: "e4b7d8e0-2007-4ead-bf26-96b80cbeeabe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 14:21:23.032107 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:23.032013 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:23.032107 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:23.032069 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:23.032296 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:23.032039 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:23.032296 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:23.032145 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8mtbj" podUID="01f02794-646c-48f8-8fb1-85cd0a5f9882" Jul 09 14:21:23.032296 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:23.032196 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjc84" podUID="e4b7d8e0-2007-4ead-bf26-96b80cbeeabe" Jul 09 14:21:23.032296 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:23.032280 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vkk5f" podUID="6050faa0-7272-4fc1-8392-6f072ab52c5a" Jul 09 14:21:23.231968 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:23.231928 2582 generic.go:356] "Generic (PLEG): container finished" podID="2a720bc8-0c99-4249-801c-a6d8e49ff8f3" containerID="4a4ee14e4bedf40369a4ed0f6ef2e6701c39817a9749f90927678de1467289f5" exitCode=0 Jul 09 14:21:23.232417 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:23.232013 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8cc" event={"ID":"2a720bc8-0c99-4249-801c-a6d8e49ff8f3","Type":"ContainerDied","Data":"4a4ee14e4bedf40369a4ed0f6ef2e6701c39817a9749f90927678de1467289f5"} Jul 09 14:21:24.075219 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.075195 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n9hkg_9205fffb-e805-4302-8d1e-535e1874e263/dns-node-resolver/0.log" Jul 09 14:21:24.468498 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.468477 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-93.ec2.internal" event="NodeReady" Jul 09 14:21:24.468919 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.468657 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 14:21:24.518573 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.518541 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7649c48849-mdnn4"] Jul 09 14:21:24.522226 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.522203 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.524803 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.524783 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 14:21:24.525132 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.525116 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 14:21:24.525263 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.525246 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-khbqc\"" Jul 09 14:21:24.525582 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.525568 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 14:21:24.529567 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.529545 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7w4gm"] Jul 09 14:21:24.530136 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.530116 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 14:21:24.533030 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.533014 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ppm58"] Jul 09 14:21:24.533149 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.533136 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:24.536135 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.536119 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 14:21:24.536278 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.536262 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ppm58" Jul 09 14:21:24.536508 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.536402 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 14:21:24.536637 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.536592 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xxvxp\"" Jul 09 14:21:24.539746 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.539016 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7649c48849-mdnn4"] Jul 09 14:21:24.539746 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.539263 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 14:21:24.539746 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.539493 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 14:21:24.539746 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.539540 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-n5vbv\"" Jul 09 14:21:24.539746 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.539633 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 14:21:24.553653 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.553633 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7w4gm"] Jul 09 14:21:24.554197 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.554179 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ppm58"] Jul 09 14:21:24.559599 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.559580 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ddv6d"] Jul 09 14:21:24.562982 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.562966 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:24.565844 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.565825 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 14:21:24.565960 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.565827 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 14:21:24.566134 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.566119 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 14:21:24.566243 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.566224 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lsg92\"" Jul 09 14:21:24.567061 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.567043 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 14:21:24.577547 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.577527 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ddv6d"] Jul 09 14:21:24.589972 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.589952 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84a2ff14-a459-43b7-a980-c7aa6ed765fb-ca-trust-extracted\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.590091 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.589981 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-certificates\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.590091 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.590003 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdhh6\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-kube-api-access-fdhh6\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.590091 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.590048 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.590091 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.590076 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84a2ff14-a459-43b7-a980-c7aa6ed765fb-trusted-ca\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.590250 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.590158 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84a2ff14-a459-43b7-a980-c7aa6ed765fb-image-registry-private-configuration\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.590250 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.590194 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84a2ff14-a459-43b7-a980-c7aa6ed765fb-installation-pull-secrets\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.590250 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.590211 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-bound-sa-token\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.691192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691124 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp9v5\" (UniqueName: \"kubernetes.io/projected/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-api-access-dp9v5\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:24.691192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691159 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.691192 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691181 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84a2ff14-a459-43b7-a980-c7aa6ed765fb-trusted-ca\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.691432 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691237 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq5gh\" (UniqueName: \"kubernetes.io/projected/c405f127-dba7-4bb3-a573-a580ef9c9ad3-kube-api-access-bq5gh\") pod \"ingress-canary-ppm58\" (UID: \"c405f127-dba7-4bb3-a573-a580ef9c9ad3\") " pod="openshift-ingress-canary/ingress-canary-ppm58" Jul 09 14:21:24.691432 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:24.691289 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:21:24.691432 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:24.691303 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7649c48849-mdnn4: secret "image-registry-tls" not found Jul 09 14:21:24.691432 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:24.691351 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls podName:84a2ff14-a459-43b7-a980-c7aa6ed765fb nodeName:}" failed. No retries permitted until 2026-07-09 14:21:25.191335558 +0000 UTC m=+35.557429923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls") pod "image-registry-7649c48849-mdnn4" (UID: "84a2ff14-a459-43b7-a980-c7aa6ed765fb") : secret "image-registry-tls" not found Jul 09 14:21:24.691432 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691371 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca7b3983-3431-45cb-a7b5-595e78ee3da4-data-volume\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:24.691432 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691388 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:24.691432 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691417 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84a2ff14-a459-43b7-a980-c7aa6ed765fb-image-registry-private-configuration\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.691757 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691442 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3599785b-ae2d-4391-afb5-b6b50594143c-tmp-dir\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:24.691757 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691476 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c405f127-dba7-4bb3-a573-a580ef9c9ad3-cert\") pod \"ingress-canary-ppm58\" (UID: \"c405f127-dba7-4bb3-a573-a580ef9c9ad3\") " pod="openshift-ingress-canary/ingress-canary-ppm58" Jul 09 14:21:24.691757 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691516 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84a2ff14-a459-43b7-a980-c7aa6ed765fb-installation-pull-secrets\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.691757 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691541 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-bound-sa-token\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.691757 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691569 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ca7b3983-3431-45cb-a7b5-595e78ee3da4-crio-socket\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:24.691757 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691590 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:24.691757 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691619 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84a2ff14-a459-43b7-a980-c7aa6ed765fb-ca-trust-extracted\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.691757 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691641 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn9qk\" (UniqueName: \"kubernetes.io/projected/3599785b-ae2d-4391-afb5-b6b50594143c-kube-api-access-wn9qk\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:24.691757 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691668 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-certificates\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.691757 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691702 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdhh6\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-kube-api-access-fdhh6\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.691757 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691729 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3599785b-ae2d-4391-afb5-b6b50594143c-config-volume\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:24.691757 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.691754 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:24.692246 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.692140 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84a2ff14-a459-43b7-a980-c7aa6ed765fb-ca-trust-extracted\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.692246 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.692181 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84a2ff14-a459-43b7-a980-c7aa6ed765fb-trusted-ca\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.692518 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.692502 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-certificates\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.695748 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.695727 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84a2ff14-a459-43b7-a980-c7aa6ed765fb-image-registry-private-configuration\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.695851 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.695726 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84a2ff14-a459-43b7-a980-c7aa6ed765fb-installation-pull-secrets\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.710821 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.710795 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdhh6\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-kube-api-access-fdhh6\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.712239 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.712220 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-bound-sa-token\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:24.792565 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.792526 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq5gh\" (UniqueName: \"kubernetes.io/projected/c405f127-dba7-4bb3-a573-a580ef9c9ad3-kube-api-access-bq5gh\") pod \"ingress-canary-ppm58\" (UID: \"c405f127-dba7-4bb3-a573-a580ef9c9ad3\") " pod="openshift-ingress-canary/ingress-canary-ppm58" Jul 09 14:21:24.792700 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.792627 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca7b3983-3431-45cb-a7b5-595e78ee3da4-data-volume\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:24.792700 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.792651 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:24.792700 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.792674 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3599785b-ae2d-4391-afb5-b6b50594143c-tmp-dir\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:24.792700 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.792694 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c405f127-dba7-4bb3-a573-a580ef9c9ad3-cert\") pod \"ingress-canary-ppm58\" (UID: \"c405f127-dba7-4bb3-a573-a580ef9c9ad3\") " pod="openshift-ingress-canary/ingress-canary-ppm58" Jul 09 14:21:24.792852 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.792742 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ca7b3983-3431-45cb-a7b5-595e78ee3da4-crio-socket\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:24.792852 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.792775 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:24.792852 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.792806 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn9qk\" (UniqueName: \"kubernetes.io/projected/3599785b-ae2d-4391-afb5-b6b50594143c-kube-api-access-wn9qk\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:24.792852 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:24.792777 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:21:24.792852 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.792844 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3599785b-ae2d-4391-afb5-b6b50594143c-config-volume\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:24.793135 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.792894 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:24.793135 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:24.792913 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:21:24.793135 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.792918 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp9v5\" (UniqueName: \"kubernetes.io/projected/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-api-access-dp9v5\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:24.793135 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:24.792959 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls podName:ca7b3983-3431-45cb-a7b5-595e78ee3da4 nodeName:}" failed. No retries permitted until 2026-07-09 14:21:25.292930661 +0000 UTC m=+35.659025044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ddv6d" (UID: "ca7b3983-3431-45cb-a7b5-595e78ee3da4") : secret "insights-runtime-extractor-tls" not found Jul 09 14:21:24.793135 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.792986 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ca7b3983-3431-45cb-a7b5-595e78ee3da4-crio-socket\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:24.793135 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.793046 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca7b3983-3431-45cb-a7b5-595e78ee3da4-data-volume\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:24.793135 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.793048 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3599785b-ae2d-4391-afb5-b6b50594143c-tmp-dir\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:24.793135 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:24.793071 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls podName:3599785b-ae2d-4391-afb5-b6b50594143c nodeName:}" failed. No retries permitted until 2026-07-09 14:21:25.293057357 +0000 UTC m=+35.659151742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls") pod "dns-default-7w4gm" (UID: "3599785b-ae2d-4391-afb5-b6b50594143c") : secret "dns-default-metrics-tls" not found Jul 09 14:21:24.793135 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:24.793117 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:21:24.793488 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:24.793171 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm podName:ca7b3983-3431-45cb-a7b5-595e78ee3da4 nodeName:}" failed. No retries permitted until 2026-07-09 14:21:25.29315019 +0000 UTC m=+35.659244577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ddv6d" (UID: "ca7b3983-3431-45cb-a7b5-595e78ee3da4") : configmap "kube-rbac-proxy" not found Jul 09 14:21:24.793488 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.793360 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3599785b-ae2d-4391-afb5-b6b50594143c-config-volume\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:24.795425 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.795406 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c405f127-dba7-4bb3-a573-a580ef9c9ad3-cert\") pod \"ingress-canary-ppm58\" (UID: \"c405f127-dba7-4bb3-a573-a580ef9c9ad3\") " pod="openshift-ingress-canary/ingress-canary-ppm58" Jul 09 14:21:24.801291 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.801271 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn9qk\" (UniqueName: \"kubernetes.io/projected/3599785b-ae2d-4391-afb5-b6b50594143c-kube-api-access-wn9qk\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:24.801396 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.801383 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp9v5\" (UniqueName: \"kubernetes.io/projected/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-api-access-dp9v5\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:24.801954 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.801937 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq5gh\" (UniqueName: \"kubernetes.io/projected/c405f127-dba7-4bb3-a573-a580ef9c9ad3-kube-api-access-bq5gh\") pod \"ingress-canary-ppm58\" (UID: \"c405f127-dba7-4bb3-a573-a580ef9c9ad3\") " pod="openshift-ingress-canary/ingress-canary-ppm58" Jul 09 14:21:24.850385 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.850358 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ppm58" Jul 09 14:21:24.997680 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:24.997651 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ppm58"] Jul 09 14:21:25.000677 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:21:25.000653 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc405f127_dba7_4bb3_a573_a580ef9c9ad3.slice/crio-d93694052e003b9f3aa1ec07f32be2231fdcf1b55794c46fc9978934b233ded4 WatchSource:0}: Error finding container d93694052e003b9f3aa1ec07f32be2231fdcf1b55794c46fc9978934b233ded4: Status 404 returned error can't find the container with id d93694052e003b9f3aa1ec07f32be2231fdcf1b55794c46fc9978934b233ded4 Jul 09 14:21:25.032149 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.032124 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:25.032236 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.032220 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:25.032276 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.032247 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:25.034593 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.034577 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bcf26\"" Jul 09 14:21:25.034593 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.034599 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 14:21:25.034753 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.034684 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 14:21:25.034753 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.034687 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d4gsp\"" Jul 09 14:21:25.034851 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.034687 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 14:21:25.034851 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.034793 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 14:21:25.074735 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.074714 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ffxnv_1e64c211-f85a-4e12-8034-67a6c1e1bb88/node-ca/0.log" Jul 09 14:21:25.196979 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.196943 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:25.197157 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:25.197097 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:21:25.197157 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:25.197115 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7649c48849-mdnn4: secret "image-registry-tls" not found Jul 09 14:21:25.197248 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:25.197162 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls podName:84a2ff14-a459-43b7-a980-c7aa6ed765fb nodeName:}" failed. No retries permitted until 2026-07-09 14:21:26.197147766 +0000 UTC m=+36.563242132 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls") pod "image-registry-7649c48849-mdnn4" (UID: "84a2ff14-a459-43b7-a980-c7aa6ed765fb") : secret "image-registry-tls" not found Jul 09 14:21:25.237573 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.237492 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ppm58" event={"ID":"c405f127-dba7-4bb3-a573-a580ef9c9ad3","Type":"ContainerStarted","Data":"d93694052e003b9f3aa1ec07f32be2231fdcf1b55794c46fc9978934b233ded4"} Jul 09 14:21:25.297925 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.297892 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:25.298090 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.297937 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:25.298090 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:25.297965 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:25.298090 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:25.298034 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:21:25.298090 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:25.298045 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:21:25.298234 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:25.298095 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm podName:ca7b3983-3431-45cb-a7b5-595e78ee3da4 nodeName:}" failed. No retries permitted until 2026-07-09 14:21:26.298082417 +0000 UTC m=+36.664176782 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ddv6d" (UID: "ca7b3983-3431-45cb-a7b5-595e78ee3da4") : configmap "kube-rbac-proxy" not found Jul 09 14:21:25.298234 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:25.298096 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:21:25.298234 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:25.298107 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls podName:ca7b3983-3431-45cb-a7b5-595e78ee3da4 nodeName:}" failed. No retries permitted until 2026-07-09 14:21:26.298100734 +0000 UTC m=+36.664195098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ddv6d" (UID: "ca7b3983-3431-45cb-a7b5-595e78ee3da4") : secret "insights-runtime-extractor-tls" not found Jul 09 14:21:25.298234 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:25.298145 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls podName:3599785b-ae2d-4391-afb5-b6b50594143c nodeName:}" failed. No retries permitted until 2026-07-09 14:21:26.29813235 +0000 UTC m=+36.664226714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls") pod "dns-default-7w4gm" (UID: "3599785b-ae2d-4391-afb5-b6b50594143c") : secret "dns-default-metrics-tls" not found Jul 09 14:21:26.203385 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.203354 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:26.204299 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:26.203526 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:21:26.204299 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:26.203547 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7649c48849-mdnn4: secret "image-registry-tls" not found Jul 09 14:21:26.204299 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:26.203610 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls podName:84a2ff14-a459-43b7-a980-c7aa6ed765fb nodeName:}" failed. No retries permitted until 2026-07-09 14:21:28.203592255 +0000 UTC m=+38.569686619 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls") pod "image-registry-7649c48849-mdnn4" (UID: "84a2ff14-a459-43b7-a980-c7aa6ed765fb") : secret "image-registry-tls" not found Jul 09 14:21:26.304190 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.304160 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:26.304350 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.304206 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:26.304350 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.304237 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:26.304350 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:26.304316 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:21:26.304350 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:26.304336 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:21:26.304350 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:26.304342 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:21:26.304519 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:26.304379 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls podName:ca7b3983-3431-45cb-a7b5-595e78ee3da4 nodeName:}" failed. No retries permitted until 2026-07-09 14:21:28.304364104 +0000 UTC m=+38.670458467 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ddv6d" (UID: "ca7b3983-3431-45cb-a7b5-595e78ee3da4") : secret "insights-runtime-extractor-tls" not found Jul 09 14:21:26.304519 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:26.304401 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm podName:ca7b3983-3431-45cb-a7b5-595e78ee3da4 nodeName:}" failed. No retries permitted until 2026-07-09 14:21:28.304387109 +0000 UTC m=+38.670481473 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ddv6d" (UID: "ca7b3983-3431-45cb-a7b5-595e78ee3da4") : configmap "kube-rbac-proxy" not found Jul 09 14:21:26.304519 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:26.304412 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls podName:3599785b-ae2d-4391-afb5-b6b50594143c nodeName:}" failed. No retries permitted until 2026-07-09 14:21:28.30440677 +0000 UTC m=+38.670501133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls") pod "dns-default-7w4gm" (UID: "3599785b-ae2d-4391-afb5-b6b50594143c") : secret "dns-default-metrics-tls" not found Jul 09 14:21:26.339083 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.339054 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-cvnrz"] Jul 09 14:21:26.375486 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.375462 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-cvnrz"] Jul 09 14:21:26.375627 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.375552 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-cvnrz" Jul 09 14:21:26.378648 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.378626 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 09 14:21:26.378795 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.378675 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 09 14:21:26.379773 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.379748 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-x8pfs\"" Jul 09 14:21:26.379773 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.379762 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 09 14:21:26.379985 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.379843 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 09 14:21:26.505542 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.505461 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5fe2297a-610e-4511-a56c-5b889318b4af-signing-key\") pod \"service-ca-79f68b9f4f-cvnrz\" (UID: \"5fe2297a-610e-4511-a56c-5b889318b4af\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cvnrz" Jul 09 14:21:26.505542 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.505493 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5fe2297a-610e-4511-a56c-5b889318b4af-signing-cabundle\") pod \"service-ca-79f68b9f4f-cvnrz\" (UID: \"5fe2297a-610e-4511-a56c-5b889318b4af\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cvnrz" Jul 09 14:21:26.505738 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.505605 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pdnk\" (UniqueName: \"kubernetes.io/projected/5fe2297a-610e-4511-a56c-5b889318b4af-kube-api-access-7pdnk\") pod \"service-ca-79f68b9f4f-cvnrz\" (UID: \"5fe2297a-610e-4511-a56c-5b889318b4af\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cvnrz" Jul 09 14:21:26.606171 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.606130 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pdnk\" (UniqueName: \"kubernetes.io/projected/5fe2297a-610e-4511-a56c-5b889318b4af-kube-api-access-7pdnk\") pod \"service-ca-79f68b9f4f-cvnrz\" (UID: \"5fe2297a-610e-4511-a56c-5b889318b4af\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cvnrz" Jul 09 14:21:26.606337 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.606268 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5fe2297a-610e-4511-a56c-5b889318b4af-signing-key\") pod \"service-ca-79f68b9f4f-cvnrz\" (UID: \"5fe2297a-610e-4511-a56c-5b889318b4af\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cvnrz" Jul 09 14:21:26.607071 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.606292 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5fe2297a-610e-4511-a56c-5b889318b4af-signing-cabundle\") pod \"service-ca-79f68b9f4f-cvnrz\" (UID: \"5fe2297a-610e-4511-a56c-5b889318b4af\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cvnrz" Jul 09 14:21:26.607622 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.607602 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5fe2297a-610e-4511-a56c-5b889318b4af-signing-cabundle\") pod \"service-ca-79f68b9f4f-cvnrz\" (UID: \"5fe2297a-610e-4511-a56c-5b889318b4af\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cvnrz" Jul 09 14:21:26.609599 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.609572 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5fe2297a-610e-4511-a56c-5b889318b4af-signing-key\") pod \"service-ca-79f68b9f4f-cvnrz\" (UID: \"5fe2297a-610e-4511-a56c-5b889318b4af\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cvnrz" Jul 09 14:21:26.621614 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.621587 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pdnk\" (UniqueName: \"kubernetes.io/projected/5fe2297a-610e-4511-a56c-5b889318b4af-kube-api-access-7pdnk\") pod \"service-ca-79f68b9f4f-cvnrz\" (UID: \"5fe2297a-610e-4511-a56c-5b889318b4af\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cvnrz" Jul 09 14:21:26.686109 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.685842 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-cvnrz" Jul 09 14:21:26.864467 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:26.864208 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-cvnrz"] Jul 09 14:21:26.877725 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:21:26.877696 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe2297a_610e_4511_a56c_5b889318b4af.slice/crio-57aa7a6287df07202dc30565adee49e9069cbc01daaa65c292013b49f9b1914b WatchSource:0}: Error finding container 57aa7a6287df07202dc30565adee49e9069cbc01daaa65c292013b49f9b1914b: Status 404 returned error can't find the container with id 57aa7a6287df07202dc30565adee49e9069cbc01daaa65c292013b49f9b1914b Jul 09 14:21:27.244512 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:27.244478 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-cvnrz" event={"ID":"5fe2297a-610e-4511-a56c-5b889318b4af","Type":"ContainerStarted","Data":"57aa7a6287df07202dc30565adee49e9069cbc01daaa65c292013b49f9b1914b"} Jul 09 14:21:28.222188 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:28.222141 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:28.222451 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:28.222431 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:21:28.222527 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:28.222455 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7649c48849-mdnn4: secret "image-registry-tls" not found Jul 09 14:21:28.222527 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:28.222524 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls podName:84a2ff14-a459-43b7-a980-c7aa6ed765fb nodeName:}" failed. No retries permitted until 2026-07-09 14:21:32.222505749 +0000 UTC m=+42.588600127 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls") pod "image-registry-7649c48849-mdnn4" (UID: "84a2ff14-a459-43b7-a980-c7aa6ed765fb") : secret "image-registry-tls" not found Jul 09 14:21:28.323795 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:28.322898 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:28.323795 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:28.323023 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:28.323795 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:28.323073 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:28.323795 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:28.323214 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:21:28.323795 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:28.323276 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls podName:3599785b-ae2d-4391-afb5-b6b50594143c nodeName:}" failed. No retries permitted until 2026-07-09 14:21:32.323257042 +0000 UTC m=+42.689351427 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls") pod "dns-default-7w4gm" (UID: "3599785b-ae2d-4391-afb5-b6b50594143c") : secret "dns-default-metrics-tls" not found Jul 09 14:21:28.323795 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:28.323327 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:21:28.323795 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:28.323361 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm podName:ca7b3983-3431-45cb-a7b5-595e78ee3da4 nodeName:}" failed. No retries permitted until 2026-07-09 14:21:32.323349869 +0000 UTC m=+42.689444247 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ddv6d" (UID: "ca7b3983-3431-45cb-a7b5-595e78ee3da4") : configmap "kube-rbac-proxy" not found Jul 09 14:21:28.323795 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:28.323423 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:21:28.323795 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:28.323453 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls podName:ca7b3983-3431-45cb-a7b5-595e78ee3da4 nodeName:}" failed. No retries permitted until 2026-07-09 14:21:32.323443345 +0000 UTC m=+42.689537724 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ddv6d" (UID: "ca7b3983-3431-45cb-a7b5-595e78ee3da4") : secret "insights-runtime-extractor-tls" not found Jul 09 14:21:30.254701 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:30.254660 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ppm58" event={"ID":"c405f127-dba7-4bb3-a573-a580ef9c9ad3","Type":"ContainerStarted","Data":"6dc8578cec9fed30cdce5160ae9c6c692c64b54b8440de50ecab9d7e4f525a95"} Jul 09 14:21:30.256473 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:30.256445 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-cvnrz" event={"ID":"5fe2297a-610e-4511-a56c-5b889318b4af","Type":"ContainerStarted","Data":"a144ad899abda370bfaaf001502fefd8a0f08911d3309e37209d9651a9909a35"} Jul 09 14:21:30.272286 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:30.272231 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ppm58" podStartSLOduration=1.880823193 podStartE2EDuration="6.272211298s" totalImagesPullingTime="4.391388105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:30.271611888 +0000 UTC m=+40.637706276" watchObservedRunningTime="2026-07-09 14:21:30.272211298 +0000 UTC m=+40.638305686" Jul 09 14:21:32.256669 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:32.256482 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:32.257067 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:32.256645 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 14:21:32.257067 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:32.256745 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7649c48849-mdnn4: secret "image-registry-tls" not found Jul 09 14:21:32.257067 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:32.256801 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls podName:84a2ff14-a459-43b7-a980-c7aa6ed765fb nodeName:}" failed. No retries permitted until 2026-07-09 14:21:40.256784967 +0000 UTC m=+50.622879331 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls") pod "image-registry-7649c48849-mdnn4" (UID: "84a2ff14-a459-43b7-a980-c7aa6ed765fb") : secret "image-registry-tls" not found Jul 09 14:21:32.269846 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:32.269808 2582 generic.go:356] "Generic (PLEG): container finished" podID="2a720bc8-0c99-4249-801c-a6d8e49ff8f3" containerID="d651496510079c4edcb1c2ee72bdccc21fd9cf9cffd15e1437e5ed64220ae873" exitCode=0 Jul 09 14:21:32.269966 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:32.269864 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8cc" event={"ID":"2a720bc8-0c99-4249-801c-a6d8e49ff8f3","Type":"ContainerDied","Data":"d651496510079c4edcb1c2ee72bdccc21fd9cf9cffd15e1437e5ed64220ae873"} Jul 09 14:21:32.297045 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:32.296960 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-cvnrz" podStartSLOduration=3.778795052 podStartE2EDuration="6.296940683s" totalImagesPullingTime="2.518145631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:21:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:30.289089342 +0000 UTC m=+40.655183750" watchObservedRunningTime="2026-07-09 14:21:32.296940683 +0000 UTC m=+42.663035069" Jul 09 14:21:32.357194 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:32.357165 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:32.357320 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:32.357298 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:21:32.357379 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:32.357361 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls podName:ca7b3983-3431-45cb-a7b5-595e78ee3da4 nodeName:}" failed. No retries permitted until 2026-07-09 14:21:40.357342387 +0000 UTC m=+50.723436752 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ddv6d" (UID: "ca7b3983-3431-45cb-a7b5-595e78ee3da4") : secret "insights-runtime-extractor-tls" not found Jul 09 14:21:32.357425 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:32.357391 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:32.357492 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:32.357474 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:32.357573 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:32.357560 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:21:32.357623 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:32.357608 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm podName:ca7b3983-3431-45cb-a7b5-595e78ee3da4 nodeName:}" failed. No retries permitted until 2026-07-09 14:21:40.35759432 +0000 UTC m=+50.723688699 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ddv6d" (UID: "ca7b3983-3431-45cb-a7b5-595e78ee3da4") : configmap "kube-rbac-proxy" not found Jul 09 14:21:32.357689 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:32.357676 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 14:21:32.357743 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:32.357709 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls podName:3599785b-ae2d-4391-afb5-b6b50594143c nodeName:}" failed. No retries permitted until 2026-07-09 14:21:40.357698823 +0000 UTC m=+50.723793187 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls") pod "dns-default-7w4gm" (UID: "3599785b-ae2d-4391-afb5-b6b50594143c") : secret "dns-default-metrics-tls" not found Jul 09 14:21:33.275309 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:33.275275 2582 generic.go:356] "Generic (PLEG): container finished" podID="2a720bc8-0c99-4249-801c-a6d8e49ff8f3" containerID="3eea50edd62eec00362029ce13846a5c7146902a787d052a826b2b35a586a3d6" exitCode=0 Jul 09 14:21:33.275665 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:33.275315 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8cc" event={"ID":"2a720bc8-0c99-4249-801c-a6d8e49ff8f3","Type":"ContainerDied","Data":"3eea50edd62eec00362029ce13846a5c7146902a787d052a826b2b35a586a3d6"} Jul 09 14:21:34.281682 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:34.281645 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dz8cc" event={"ID":"2a720bc8-0c99-4249-801c-a6d8e49ff8f3","Type":"ContainerStarted","Data":"de59716b8a613df4d2eb313791733e15316e8f28f3e14ec4009fa56ca98e989c"} Jul 09 14:21:34.304409 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:34.304342 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dz8cc" podStartSLOduration=7.407845076 podStartE2EDuration="44.304321354s" totalImagesPullingTime="36.896476278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:20:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:34.303914611 +0000 UTC m=+44.670008997" watchObservedRunningTime="2026-07-09 14:21:34.304321354 +0000 UTC m=+44.670415743" Jul 09 14:21:40.328718 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:40.328679 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:40.332753 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:40.332731 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls\") pod \"image-registry-7649c48849-mdnn4\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:40.429527 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:40.429489 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:40.429734 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:40.429542 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:40.429734 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:40.429571 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:40.429734 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:40.429642 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:21:40.429734 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:21:40.429726 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls podName:ca7b3983-3431-45cb-a7b5-595e78ee3da4 nodeName:}" failed. No retries permitted until 2026-07-09 14:21:56.429704712 +0000 UTC m=+66.795799100 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ddv6d" (UID: "ca7b3983-3431-45cb-a7b5-595e78ee3da4") : secret "insights-runtime-extractor-tls" not found Jul 09 14:21:40.430188 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:40.430167 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca7b3983-3431-45cb-a7b5-595e78ee3da4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:40.431134 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:40.431112 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:40.432096 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:40.432077 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3599785b-ae2d-4391-afb5-b6b50594143c-metrics-tls\") pod \"dns-default-7w4gm\" (UID: \"3599785b-ae2d-4391-afb5-b6b50594143c\") " pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:40.445360 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:40.445337 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:40.597130 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:40.596928 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7649c48849-mdnn4"] Jul 09 14:21:40.615809 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:40.615786 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7w4gm"] Jul 09 14:21:40.618729 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:21:40.618696 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3599785b_ae2d_4391_afb5_b6b50594143c.slice/crio-4a8780d6faeac49f2b721dec48f9aae6d91317eafdabf25e3219ae54da0becec WatchSource:0}: Error finding container 4a8780d6faeac49f2b721dec48f9aae6d91317eafdabf25e3219ae54da0becec: Status 404 returned error can't find the container with id 4a8780d6faeac49f2b721dec48f9aae6d91317eafdabf25e3219ae54da0becec Jul 09 14:21:41.298498 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:41.298462 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7w4gm" event={"ID":"3599785b-ae2d-4391-afb5-b6b50594143c","Type":"ContainerStarted","Data":"4a8780d6faeac49f2b721dec48f9aae6d91317eafdabf25e3219ae54da0becec"} Jul 09 14:21:41.299886 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:41.299846 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7649c48849-mdnn4" event={"ID":"84a2ff14-a459-43b7-a980-c7aa6ed765fb","Type":"ContainerStarted","Data":"ad9feb2274abc872f94da3c1ad140ee326df9dbdc27c0055d441e8a23e513132"} Jul 09 14:21:41.300024 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:41.299906 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7649c48849-mdnn4" event={"ID":"84a2ff14-a459-43b7-a980-c7aa6ed765fb","Type":"ContainerStarted","Data":"527c5be66a43e37c421ce6e3bc9d4c19c96d45fd36a7d7bf0b2ce91f674f0723"} Jul 09 14:21:41.300024 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:41.299993 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:21:41.325519 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:41.325465 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7649c48849-mdnn4" podStartSLOduration=50.325444563 podStartE2EDuration="50.325444563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:41.324099954 +0000 UTC m=+51.690194341" watchObservedRunningTime="2026-07-09 14:21:41.325444563 +0000 UTC m=+51.691538950" Jul 09 14:21:43.306241 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:43.306212 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7w4gm" event={"ID":"3599785b-ae2d-4391-afb5-b6b50594143c","Type":"ContainerStarted","Data":"141e692707df3a8141806ebcc8dfd75504f76dbe3b6d11c59389e84647c168d9"} Jul 09 14:21:44.310933 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:44.310894 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7w4gm" event={"ID":"3599785b-ae2d-4391-afb5-b6b50594143c","Type":"ContainerStarted","Data":"891a55073a13169c4caca8f54da7cedf4ffd7f44d3f612cf21e41f5add5aa4b6"} Jul 09 14:21:44.311401 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:44.311010 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:44.328963 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:44.328919 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7w4gm" podStartSLOduration=17.781933269 podStartE2EDuration="20.328904528s" totalImagesPullingTime="2.546971259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:44.327667811 +0000 UTC m=+54.693762197" watchObservedRunningTime="2026-07-09 14:21:44.328904528 +0000 UTC m=+54.694998914" Jul 09 14:21:47.440344 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.440312 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7649c48849-mdnn4"] Jul 09 14:21:47.454650 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.454620 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w"] Jul 09 14:21:47.457987 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.457969 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8"] Jul 09 14:21:47.458104 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.458085 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" Jul 09 14:21:47.460488 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.460467 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775d7978-z55xt"] Jul 09 14:21:47.460609 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.460588 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.461134 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.461111 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 09 14:21:47.461134 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.461131 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 14:21:47.461305 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.461112 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 14:21:47.461305 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.461172 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 14:21:47.462954 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.462936 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775d7978-z55xt" Jul 09 14:21:47.463526 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.463508 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 09 14:21:47.464285 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.464265 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 09 14:21:47.464504 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.464486 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 09 14:21:47.465435 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.465395 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4dgjp\"" Jul 09 14:21:47.465820 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.465805 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 09 14:21:47.466009 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.465989 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 09 14:21:47.468207 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.468189 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w"] Jul 09 14:21:47.473172 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.473150 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775d7978-z55xt"] Jul 09 14:21:47.479702 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.479675 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8"] Jul 09 14:21:47.479908 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.479866 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/23c85b14-376e-461c-bfb6-ce17b92cd4f0-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d48b57db-nvr5w\" (UID: \"23c85b14-376e-461c-bfb6-ce17b92cd4f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" Jul 09 14:21:47.480021 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.479938 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gskzp\" (UniqueName: \"kubernetes.io/projected/8be7b838-5eab-432f-a10f-b78af7ae56e3-kube-api-access-gskzp\") pod \"managed-serviceaccount-addon-agent-7c775d7978-z55xt\" (UID: \"8be7b838-5eab-432f-a10f-b78af7ae56e3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775d7978-z55xt" Jul 09 14:21:47.480021 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.479969 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/023d01a2-4656-4fd2-83cd-1f600ba40cec-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.480118 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.480029 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/023d01a2-4656-4fd2-83cd-1f600ba40cec-additional-service-ca\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.480118 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.480091 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/023d01a2-4656-4fd2-83cd-1f600ba40cec-hub\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.480202 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.480120 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/023d01a2-4656-4fd2-83cd-1f600ba40cec-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.480202 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.480140 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23c85b14-376e-461c-bfb6-ce17b92cd4f0-tmp\") pod \"klusterlet-addon-workmgr-5d48b57db-nvr5w\" (UID: \"23c85b14-376e-461c-bfb6-ce17b92cd4f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" Jul 09 14:21:47.480279 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.480199 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8be7b838-5eab-432f-a10f-b78af7ae56e3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c775d7978-z55xt\" (UID: \"8be7b838-5eab-432f-a10f-b78af7ae56e3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775d7978-z55xt" Jul 09 14:21:47.480279 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.480228 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqrvt\" (UniqueName: \"kubernetes.io/projected/23c85b14-376e-461c-bfb6-ce17b92cd4f0-kube-api-access-cqrvt\") pod \"klusterlet-addon-workmgr-5d48b57db-nvr5w\" (UID: \"23c85b14-376e-461c-bfb6-ce17b92cd4f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" Jul 09 14:21:47.480340 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.480294 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/023d01a2-4656-4fd2-83cd-1f600ba40cec-ca\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.480340 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.480313 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl8br\" (UniqueName: \"kubernetes.io/projected/023d01a2-4656-4fd2-83cd-1f600ba40cec-kube-api-access-kl8br\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.537457 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.537419 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bf6c9b6b-gh4ns"] Jul 09 14:21:47.540243 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.540220 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.555509 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.555489 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bf6c9b6b-gh4ns"] Jul 09 14:21:47.581049 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581022 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23c85b14-376e-461c-bfb6-ce17b92cd4f0-tmp\") pod \"klusterlet-addon-workmgr-5d48b57db-nvr5w\" (UID: \"23c85b14-376e-461c-bfb6-ce17b92cd4f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" Jul 09 14:21:47.581173 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581052 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8be7b838-5eab-432f-a10f-b78af7ae56e3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c775d7978-z55xt\" (UID: \"8be7b838-5eab-432f-a10f-b78af7ae56e3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775d7978-z55xt" Jul 09 14:21:47.581173 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581074 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-ca-trust-extracted\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.581173 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581096 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqrvt\" (UniqueName: \"kubernetes.io/projected/23c85b14-376e-461c-bfb6-ce17b92cd4f0-kube-api-access-cqrvt\") pod \"klusterlet-addon-workmgr-5d48b57db-nvr5w\" (UID: \"23c85b14-376e-461c-bfb6-ce17b92cd4f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" Jul 09 14:21:47.581173 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581141 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-trusted-ca\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.581370 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581171 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-installation-pull-secrets\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.581370 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581199 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-bound-sa-token\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.581370 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581260 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/023d01a2-4656-4fd2-83cd-1f600ba40cec-ca\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.581370 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581290 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl8br\" (UniqueName: \"kubernetes.io/projected/023d01a2-4656-4fd2-83cd-1f600ba40cec-kube-api-access-kl8br\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.581370 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581348 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-registry-tls\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.581803 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581378 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/23c85b14-376e-461c-bfb6-ce17b92cd4f0-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d48b57db-nvr5w\" (UID: \"23c85b14-376e-461c-bfb6-ce17b92cd4f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" Jul 09 14:21:47.581803 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581405 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-image-registry-private-configuration\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.581803 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581379 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23c85b14-376e-461c-bfb6-ce17b92cd4f0-tmp\") pod \"klusterlet-addon-workmgr-5d48b57db-nvr5w\" (UID: \"23c85b14-376e-461c-bfb6-ce17b92cd4f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" Jul 09 14:21:47.581803 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581441 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gskzp\" (UniqueName: \"kubernetes.io/projected/8be7b838-5eab-432f-a10f-b78af7ae56e3-kube-api-access-gskzp\") pod \"managed-serviceaccount-addon-agent-7c775d7978-z55xt\" (UID: \"8be7b838-5eab-432f-a10f-b78af7ae56e3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775d7978-z55xt" Jul 09 14:21:47.581803 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581488 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/023d01a2-4656-4fd2-83cd-1f600ba40cec-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.581803 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581635 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g48l4\" (UniqueName: \"kubernetes.io/projected/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-kube-api-access-g48l4\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.581803 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581670 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-registry-certificates\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.581803 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/023d01a2-4656-4fd2-83cd-1f600ba40cec-additional-service-ca\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.581803 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581774 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/023d01a2-4656-4fd2-83cd-1f600ba40cec-hub\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.582439 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.581820 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/023d01a2-4656-4fd2-83cd-1f600ba40cec-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.582439 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.582276 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/023d01a2-4656-4fd2-83cd-1f600ba40cec-additional-service-ca\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.584217 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.584193 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/023d01a2-4656-4fd2-83cd-1f600ba40cec-ca\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.584446 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.584423 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/23c85b14-376e-461c-bfb6-ce17b92cd4f0-klusterlet-config\") pod \"klusterlet-addon-workmgr-5d48b57db-nvr5w\" (UID: \"23c85b14-376e-461c-bfb6-ce17b92cd4f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" Jul 09 14:21:47.584556 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.584440 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8be7b838-5eab-432f-a10f-b78af7ae56e3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c775d7978-z55xt\" (UID: \"8be7b838-5eab-432f-a10f-b78af7ae56e3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775d7978-z55xt" Jul 09 14:21:47.584659 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.584637 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/023d01a2-4656-4fd2-83cd-1f600ba40cec-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.584726 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.584700 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/023d01a2-4656-4fd2-83cd-1f600ba40cec-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.584796 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.584775 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/023d01a2-4656-4fd2-83cd-1f600ba40cec-hub\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.592664 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.592636 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl8br\" (UniqueName: \"kubernetes.io/projected/023d01a2-4656-4fd2-83cd-1f600ba40cec-kube-api-access-kl8br\") pod \"cluster-proxy-proxy-agent-55f6b67797-sqfb8\" (UID: \"023d01a2-4656-4fd2-83cd-1f600ba40cec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.594856 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.593052 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqrvt\" (UniqueName: \"kubernetes.io/projected/23c85b14-376e-461c-bfb6-ce17b92cd4f0-kube-api-access-cqrvt\") pod \"klusterlet-addon-workmgr-5d48b57db-nvr5w\" (UID: \"23c85b14-376e-461c-bfb6-ce17b92cd4f0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" Jul 09 14:21:47.596902 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.596857 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gskzp\" (UniqueName: \"kubernetes.io/projected/8be7b838-5eab-432f-a10f-b78af7ae56e3-kube-api-access-gskzp\") pod \"managed-serviceaccount-addon-agent-7c775d7978-z55xt\" (UID: \"8be7b838-5eab-432f-a10f-b78af7ae56e3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775d7978-z55xt" Jul 09 14:21:47.683066 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.683027 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-ca-trust-extracted\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.683066 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.683063 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-trusted-ca\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.683325 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.683092 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-installation-pull-secrets\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.683325 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.683218 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-bound-sa-token\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.683430 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.683323 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-registry-tls\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.683430 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.683355 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-image-registry-private-configuration\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.683430 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.683396 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g48l4\" (UniqueName: \"kubernetes.io/projected/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-kube-api-access-g48l4\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.683430 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.683405 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-ca-trust-extracted\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.683430 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.683422 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-registry-certificates\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.684248 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.684219 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-registry-certificates\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.684426 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.684393 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-trusted-ca\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.685596 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.685568 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-installation-pull-secrets\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.685844 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.685823 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-image-registry-private-configuration\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.685939 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.685846 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-registry-tls\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.691658 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.691590 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-bound-sa-token\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.691817 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.691795 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g48l4\" (UniqueName: \"kubernetes.io/projected/e4968b8a-08db-43ce-8c1f-0a53a5ad164b-kube-api-access-g48l4\") pod \"image-registry-bf6c9b6b-gh4ns\" (UID: \"e4968b8a-08db-43ce-8c1f-0a53a5ad164b\") " pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.769815 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.769780 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" Jul 09 14:21:47.783520 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.783491 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" Jul 09 14:21:47.788503 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.788482 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775d7978-z55xt" Jul 09 14:21:47.857394 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.857015 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:47.930926 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.930863 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w"] Jul 09 14:21:47.936152 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:21:47.936124 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23c85b14_376e_461c_bfb6_ce17b92cd4f0.slice/crio-8744702ccdb818592397f303c0d85852399a60e5e8973b929bc0196b77f0c4ba WatchSource:0}: Error finding container 8744702ccdb818592397f303c0d85852399a60e5e8973b929bc0196b77f0c4ba: Status 404 returned error can't find the container with id 8744702ccdb818592397f303c0d85852399a60e5e8973b929bc0196b77f0c4ba Jul 09 14:21:47.951046 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.950986 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775d7978-z55xt"] Jul 09 14:21:47.954624 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:47.954596 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8"] Jul 09 14:21:47.958977 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:21:47.958900 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod023d01a2_4656_4fd2_83cd_1f600ba40cec.slice/crio-3217f0343d9bf17907fe185311d7ae971e45f234f83f07824ba7e960e2ac21b4 WatchSource:0}: Error finding container 3217f0343d9bf17907fe185311d7ae971e45f234f83f07824ba7e960e2ac21b4: Status 404 returned error can't find the container with id 3217f0343d9bf17907fe185311d7ae971e45f234f83f07824ba7e960e2ac21b4 Jul 09 14:21:48.005578 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:48.005537 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bf6c9b6b-gh4ns"] Jul 09 14:21:48.009197 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:21:48.009171 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4968b8a_08db_43ce_8c1f_0a53a5ad164b.slice/crio-b0020e52b9ab484d5489925900d61011d1f4386d41439195be3e1683906a1349 WatchSource:0}: Error finding container b0020e52b9ab484d5489925900d61011d1f4386d41439195be3e1683906a1349: Status 404 returned error can't find the container with id b0020e52b9ab484d5489925900d61011d1f4386d41439195be3e1683906a1349 Jul 09 14:21:48.324709 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:48.324671 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" event={"ID":"23c85b14-376e-461c-bfb6-ce17b92cd4f0","Type":"ContainerStarted","Data":"8744702ccdb818592397f303c0d85852399a60e5e8973b929bc0196b77f0c4ba"} Jul 09 14:21:48.325743 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:48.325709 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" event={"ID":"023d01a2-4656-4fd2-83cd-1f600ba40cec","Type":"ContainerStarted","Data":"3217f0343d9bf17907fe185311d7ae971e45f234f83f07824ba7e960e2ac21b4"} Jul 09 14:21:48.327173 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:48.327145 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" event={"ID":"e4968b8a-08db-43ce-8c1f-0a53a5ad164b","Type":"ContainerStarted","Data":"fa5723b62bb9cd530416f265814ec84849bd5370acb724d4f451b534c0b67685"} Jul 09 14:21:48.327298 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:48.327193 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" event={"ID":"e4968b8a-08db-43ce-8c1f-0a53a5ad164b","Type":"ContainerStarted","Data":"b0020e52b9ab484d5489925900d61011d1f4386d41439195be3e1683906a1349"} Jul 09 14:21:48.327365 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:48.327296 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:21:48.328360 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:48.328329 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775d7978-z55xt" event={"ID":"8be7b838-5eab-432f-a10f-b78af7ae56e3","Type":"ContainerStarted","Data":"1372b6740664308344e48ddad842b12d34a16864a1aa6d7dcc55f1cf57c766f0"} Jul 09 14:21:48.348062 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:48.347989 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" podStartSLOduration=1.347974726 podStartE2EDuration="1.347974726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:21:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:48.347233885 +0000 UTC m=+58.713328273" watchObservedRunningTime="2026-07-09 14:21:48.347974726 +0000 UTC m=+58.714069112" Jul 09 14:21:50.234590 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:50.234560 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l5qkd" Jul 09 14:21:53.348503 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:53.348473 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" event={"ID":"23c85b14-376e-461c-bfb6-ce17b92cd4f0","Type":"ContainerStarted","Data":"2be37f5cfe91876b16ef6bbaec7864b70e693ff3d52a25320e72493291f7887a"} Jul 09 14:21:53.348849 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:53.348707 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" Jul 09 14:21:53.350102 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:53.350081 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" Jul 09 14:21:53.350630 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:53.350608 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" event={"ID":"023d01a2-4656-4fd2-83cd-1f600ba40cec","Type":"ContainerStarted","Data":"980d83693679bdef80a67b0b59c92ed3de3df955937cedc894df33e85fe12036"} Jul 09 14:21:53.350722 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:53.350648 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" event={"ID":"023d01a2-4656-4fd2-83cd-1f600ba40cec","Type":"ContainerStarted","Data":"e9136d09558b296292d5ec60093075d4adcf92b56aa034d610536e4bb90c08fe"} Jul 09 14:21:53.381431 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:53.381323 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5d48b57db-nvr5w" podStartSLOduration=1.100165449 podStartE2EDuration="6.381307586s" totalImagesPullingTime="5.281142137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:21:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:53.364929106 +0000 UTC m=+63.731023490" watchObservedRunningTime="2026-07-09 14:21:53.381307586 +0000 UTC m=+63.747401972" Jul 09 14:21:54.316577 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:54.316550 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7w4gm" Jul 09 14:21:54.356736 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:54.356694 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" event={"ID":"023d01a2-4656-4fd2-83cd-1f600ba40cec","Type":"ContainerStarted","Data":"a3a389c80a2f8de49b7ba0cca69130f8aa72143ec6567606cd8260fc9b31987b"} Jul 09 14:21:54.374558 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:54.374514 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55f6b67797-sqfb8" podStartSLOduration=2.121608548 podStartE2EDuration="7.37449777s" totalImagesPullingTime="5.252889222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:21:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:54.374277741 +0000 UTC m=+64.740372120" watchObservedRunningTime="2026-07-09 14:21:54.37449777 +0000 UTC m=+64.740592157" Jul 09 14:21:54.844750 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:54.844704 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:54.844955 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:54.844809 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs\") pod \"network-metrics-daemon-vkk5f\" (UID: \"6050faa0-7272-4fc1-8392-6f072ab52c5a\") " pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:54.847716 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:54.847688 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 14:21:54.847849 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:54.847688 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 14:21:54.858416 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:54.858388 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01f02794-646c-48f8-8fb1-85cd0a5f9882-original-pull-secret\") pod \"global-pull-secret-syncer-8mtbj\" (UID: \"01f02794-646c-48f8-8fb1-85cd0a5f9882\") " pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:54.858516 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:54.858388 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6050faa0-7272-4fc1-8392-6f072ab52c5a-metrics-certs\") pod \"network-metrics-daemon-vkk5f\" (UID: \"6050faa0-7272-4fc1-8392-6f072ab52c5a\") " pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:54.946193 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:54.946141 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd5km\" (UniqueName: \"kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km\") pod \"network-check-target-kjc84\" (UID: \"e4b7d8e0-2007-4ead-bf26-96b80cbeeabe\") " pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:54.949181 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:54.949161 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 14:21:54.959517 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:54.959495 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 14:21:54.969457 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:54.969432 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd5km\" (UniqueName: \"kubernetes.io/projected/e4b7d8e0-2007-4ead-bf26-96b80cbeeabe-kube-api-access-xd5km\") pod \"network-check-target-kjc84\" (UID: \"e4b7d8e0-2007-4ead-bf26-96b80cbeeabe\") " pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:55.042195 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:55.042166 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8mtbj" Jul 09 14:21:55.050953 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:55.050929 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bcf26\"" Jul 09 14:21:55.055036 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:55.055014 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d4gsp\"" Jul 09 14:21:55.058832 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:55.058811 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:21:55.063922 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:55.063898 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vkk5f" Jul 09 14:21:55.187174 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:55.187113 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8mtbj"] Jul 09 14:21:55.193251 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:21:55.193217 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01f02794_646c_48f8_8fb1_85cd0a5f9882.slice/crio-f9794885e431ea9770988803a71f65439646fbbe3713ec295f3cffc888180d4c WatchSource:0}: Error finding container f9794885e431ea9770988803a71f65439646fbbe3713ec295f3cffc888180d4c: Status 404 returned error can't find the container with id f9794885e431ea9770988803a71f65439646fbbe3713ec295f3cffc888180d4c Jul 09 14:21:55.206917 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:55.206865 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kjc84"] Jul 09 14:21:55.210141 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:21:55.210117 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4b7d8e0_2007_4ead_bf26_96b80cbeeabe.slice/crio-852b1119497a555461e88509acb7d7b8f9f399c9c92b44cb5d262986f83069f8 WatchSource:0}: Error finding container 852b1119497a555461e88509acb7d7b8f9f399c9c92b44cb5d262986f83069f8: Status 404 returned error can't find the container with id 852b1119497a555461e88509acb7d7b8f9f399c9c92b44cb5d262986f83069f8 Jul 09 14:21:55.223246 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:55.223222 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vkk5f"] Jul 09 14:21:55.225996 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:21:55.225975 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6050faa0_7272_4fc1_8392_6f072ab52c5a.slice/crio-db6eb1fa6954415e916fa62878743d727dd6df185a830a42ef03b0404f39ccf0 WatchSource:0}: Error finding container db6eb1fa6954415e916fa62878743d727dd6df185a830a42ef03b0404f39ccf0: Status 404 returned error can't find the container with id db6eb1fa6954415e916fa62878743d727dd6df185a830a42ef03b0404f39ccf0 Jul 09 14:21:55.361539 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:55.361471 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kjc84" event={"ID":"e4b7d8e0-2007-4ead-bf26-96b80cbeeabe","Type":"ContainerStarted","Data":"852b1119497a555461e88509acb7d7b8f9f399c9c92b44cb5d262986f83069f8"} Jul 09 14:21:55.362507 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:55.362484 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8mtbj" event={"ID":"01f02794-646c-48f8-8fb1-85cd0a5f9882","Type":"ContainerStarted","Data":"f9794885e431ea9770988803a71f65439646fbbe3713ec295f3cffc888180d4c"} Jul 09 14:21:55.363480 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:55.363463 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vkk5f" event={"ID":"6050faa0-7272-4fc1-8392-6f072ab52c5a","Type":"ContainerStarted","Data":"db6eb1fa6954415e916fa62878743d727dd6df185a830a42ef03b0404f39ccf0"} Jul 09 14:21:56.459909 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:56.459752 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:56.462744 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:56.462716 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca7b3983-3431-45cb-a7b5-595e78ee3da4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ddv6d\" (UID: \"ca7b3983-3431-45cb-a7b5-595e78ee3da4\") " pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:56.674496 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:56.674464 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lsg92\"" Jul 09 14:21:56.682194 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:56.682167 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ddv6d" Jul 09 14:21:56.857943 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:56.857912 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ddv6d"] Jul 09 14:21:57.374316 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:57.374229 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ddv6d" event={"ID":"ca7b3983-3431-45cb-a7b5-595e78ee3da4","Type":"ContainerStarted","Data":"7568162f95b96d24aa893571eb7ffedaa60c30025d88f8bcbdb48f977e06b663"} Jul 09 14:21:57.448963 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:57.448617 2582 patch_prober.go:28] interesting pod/image-registry-7649c48849-mdnn4 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 14:21:57.448963 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:57.448680 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7649c48849-mdnn4" podUID="84a2ff14-a459-43b7-a980-c7aa6ed765fb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:21:58.381408 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:58.381369 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vkk5f" event={"ID":"6050faa0-7272-4fc1-8392-6f072ab52c5a","Type":"ContainerStarted","Data":"b9ad294cfb8e4831947b10a1b70d108a6dc52753a81c68085302b2abb5a15da0"} Jul 09 14:21:58.381859 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:58.381442 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vkk5f" event={"ID":"6050faa0-7272-4fc1-8392-6f072ab52c5a","Type":"ContainerStarted","Data":"f1b1708d8d9923ff1a467c34554803278f39076684999d37b183623bd3e57929"} Jul 09 14:21:58.397178 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:21:58.397122 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vkk5f" podStartSLOduration=65.844619592 podStartE2EDuration="1m8.397101331s" totalImagesPullingTime="2.552481739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:21:58.39656271 +0000 UTC m=+68.762657096" watchObservedRunningTime="2026-07-09 14:21:58.397101331 +0000 UTC m=+68.763195719" Jul 09 14:22:01.466076 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.466031 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tjjqc"] Jul 09 14:22:01.548738 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.547969 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.551941 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.551897 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 14:22:01.552071 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.552031 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 14:22:01.552166 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.552146 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 14:22:01.552254 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.552239 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5nhnb\"" Jul 09 14:22:01.553919 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.553179 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 14:22:01.553919 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.553202 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 14:22:01.553919 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.553394 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 14:22:01.606187 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.606157 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.606345 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.606195 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7ae80ed-5d58-4389-85a9-0fb2171296c0-sys\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.606345 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.606237 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f7ae80ed-5d58-4389-85a9-0fb2171296c0-root\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.606345 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.606308 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-textfile\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.606345 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.606340 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w2st\" (UniqueName: \"kubernetes.io/projected/f7ae80ed-5d58-4389-85a9-0fb2171296c0-kube-api-access-7w2st\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.606513 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.606373 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.606513 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.606397 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-wtmp\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.606513 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.606417 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-tls\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.606513 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.606435 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7ae80ed-5d58-4389-85a9-0fb2171296c0-metrics-client-ca\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.707521 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.707478 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.707682 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.707530 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-wtmp\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.707682 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.707557 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-tls\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.707682 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.707588 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7ae80ed-5d58-4389-85a9-0fb2171296c0-metrics-client-ca\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.707682 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.707614 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.707682 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.707644 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7ae80ed-5d58-4389-85a9-0fb2171296c0-sys\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.707961 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.707694 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f7ae80ed-5d58-4389-85a9-0fb2171296c0-root\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.707961 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.707725 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-textfile\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.707961 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.707753 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7ae80ed-5d58-4389-85a9-0fb2171296c0-sys\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.707961 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.707764 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w2st\" (UniqueName: \"kubernetes.io/projected/f7ae80ed-5d58-4389-85a9-0fb2171296c0-kube-api-access-7w2st\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.707961 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.707793 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-wtmp\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.708210 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.708109 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-textfile\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.708210 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.708190 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.708323 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.708308 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7ae80ed-5d58-4389-85a9-0fb2171296c0-metrics-client-ca\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.708548 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.708359 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f7ae80ed-5d58-4389-85a9-0fb2171296c0-root\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.710442 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.710413 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-tls\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.710971 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.710947 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7ae80ed-5d58-4389-85a9-0fb2171296c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.717271 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.717213 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w2st\" (UniqueName: \"kubernetes.io/projected/f7ae80ed-5d58-4389-85a9-0fb2171296c0-kube-api-access-7w2st\") pod \"node-exporter-tjjqc\" (UID: \"f7ae80ed-5d58-4389-85a9-0fb2171296c0\") " pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:01.861477 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:01.861324 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tjjqc" Jul 09 14:22:04.076207 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:22:04.076177 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7ae80ed_5d58_4389_85a9_0fb2171296c0.slice/crio-c57a97ef35a45d60f1684ee49db41fc4800e65bbd6822dffbd5ba88deac0baf9 WatchSource:0}: Error finding container c57a97ef35a45d60f1684ee49db41fc4800e65bbd6822dffbd5ba88deac0baf9: Status 404 returned error can't find the container with id c57a97ef35a45d60f1684ee49db41fc4800e65bbd6822dffbd5ba88deac0baf9 Jul 09 14:22:04.408970 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:04.408858 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tjjqc" event={"ID":"f7ae80ed-5d58-4389-85a9-0fb2171296c0","Type":"ContainerStarted","Data":"c57a97ef35a45d60f1684ee49db41fc4800e65bbd6822dffbd5ba88deac0baf9"} Jul 09 14:22:04.410523 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:04.410494 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kjc84" event={"ID":"e4b7d8e0-2007-4ead-bf26-96b80cbeeabe","Type":"ContainerStarted","Data":"71fb94dc0732d142a59d828ba3c13854ff30bd034fcbeea59393fe4b913abd73"} Jul 09 14:22:04.410699 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:04.410589 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:22:04.412062 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:04.412035 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8mtbj" event={"ID":"01f02794-646c-48f8-8fb1-85cd0a5f9882","Type":"ContainerStarted","Data":"aa91827b179661df8734ca853e79e75348149092913947ca56d23d492d25904c"} Jul 09 14:22:04.426999 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:04.426946 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kjc84" podStartSLOduration=65.566146558 podStartE2EDuration="1m14.426929564s" totalImagesPullingTime="8.860783006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:22:04.426322366 +0000 UTC m=+74.792416757" watchObservedRunningTime="2026-07-09 14:22:04.426929564 +0000 UTC m=+74.793023951" Jul 09 14:22:04.443527 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:04.443480 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8mtbj" podStartSLOduration=65.567510028 podStartE2EDuration="1m14.443465809s" totalImagesPullingTime="8.875955781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:22:04.442319384 +0000 UTC m=+74.808413762" watchObservedRunningTime="2026-07-09 14:22:04.443465809 +0000 UTC m=+74.809560173" Jul 09 14:22:05.417299 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:05.417271 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tjjqc" event={"ID":"f7ae80ed-5d58-4389-85a9-0fb2171296c0","Type":"ContainerStarted","Data":"6567b8f5325f984570d6519a018513fd082ed1d56bb8533fe9243101182f0151"} Jul 09 14:22:05.418796 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:05.418770 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ddv6d" event={"ID":"ca7b3983-3431-45cb-a7b5-595e78ee3da4","Type":"ContainerStarted","Data":"39b5f0cd39184faffd960a20ba020c08b889dd5ee050a791ca28e769827edcbf"} Jul 09 14:22:06.425180 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:06.425141 2582 generic.go:356] "Generic (PLEG): container finished" podID="f7ae80ed-5d58-4389-85a9-0fb2171296c0" containerID="6567b8f5325f984570d6519a018513fd082ed1d56bb8533fe9243101182f0151" exitCode=0 Jul 09 14:22:06.425686 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:06.425273 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tjjqc" event={"ID":"f7ae80ed-5d58-4389-85a9-0fb2171296c0","Type":"ContainerDied","Data":"6567b8f5325f984570d6519a018513fd082ed1d56bb8533fe9243101182f0151"} Jul 09 14:22:06.427673 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:06.427645 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ddv6d" event={"ID":"ca7b3983-3431-45cb-a7b5-595e78ee3da4","Type":"ContainerStarted","Data":"3618c2543d058160c6d9c3dc9a94e8b3319a95c0460657c97a7286ce5a479085"} Jul 09 14:22:07.434552 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:07.434514 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tjjqc" event={"ID":"f7ae80ed-5d58-4389-85a9-0fb2171296c0","Type":"ContainerStarted","Data":"9605bf60a33bc697b9af0326a4bb3f185e5badcee4c10dc4aa4ecaaef94e573a"} Jul 09 14:22:07.434977 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:07.434616 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tjjqc" event={"ID":"f7ae80ed-5d58-4389-85a9-0fb2171296c0","Type":"ContainerStarted","Data":"176ca7b8867c4e0b66d3cfa2f94b1968bd71ba0494b31d29754155827ce7d66d"} Jul 09 14:22:07.445538 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:07.445515 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:22:07.456396 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:07.456339 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tjjqc" podStartSLOduration=5.1952888 podStartE2EDuration="6.4563192s" totalImagesPullingTime="1.180288259s" totalInitContainerRuntime="80.742141ms" isStatefulPod=true podCreationTimestamp="2026-07-09 14:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:22:07.453979645 +0000 UTC m=+77.820074056" watchObservedRunningTime="2026-07-09 14:22:07.4563192 +0000 UTC m=+77.822413586" Jul 09 14:22:09.337826 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:09.337794 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-bf6c9b6b-gh4ns" Jul 09 14:22:09.443883 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:09.443830 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ddv6d" event={"ID":"ca7b3983-3431-45cb-a7b5-595e78ee3da4","Type":"ContainerStarted","Data":"88606861ac2a45de17dabdbdd9d2ef97dda9914ba55caa70f89651b4b36af1a7"} Jul 09 14:22:09.445466 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:09.445436 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775d7978-z55xt" event={"ID":"8be7b838-5eab-432f-a10f-b78af7ae56e3","Type":"ContainerStarted","Data":"a0201c91d93285ecf813d1d022d34bae0d2210033a8e76ee88ed65f522fdcc48"} Jul 09 14:22:09.483020 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:09.482969 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775d7978-z55xt" podStartSLOduration=1.697536422 podStartE2EDuration="22.482952352s" totalImagesPullingTime="20.78541593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:21:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:22:09.48219617 +0000 UTC m=+79.848290555" watchObservedRunningTime="2026-07-09 14:22:09.482952352 +0000 UTC m=+79.849046716" Jul 09 14:22:09.483906 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:09.483853 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ddv6d" podStartSLOduration=33.786546678 podStartE2EDuration="45.483845847s" totalImagesPullingTime="11.697299169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:21:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:22:09.465674707 +0000 UTC m=+79.831769108" watchObservedRunningTime="2026-07-09 14:22:09.483845847 +0000 UTC m=+79.849940232" Jul 09 14:22:12.460640 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.460504 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7649c48849-mdnn4" podUID="84a2ff14-a459-43b7-a980-c7aa6ed765fb" containerName="registry" containerID="cri-o://ad9feb2274abc872f94da3c1ad140ee326df9dbdc27c0055d441e8a23e513132" gracePeriod=30 Jul 09 14:22:12.707197 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.707171 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:22:12.801166 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.801082 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls\") pod \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " Jul 09 14:22:12.801166 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.801113 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-bound-sa-token\") pod \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " Jul 09 14:22:12.801166 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.801161 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84a2ff14-a459-43b7-a980-c7aa6ed765fb-ca-trust-extracted\") pod \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " Jul 09 14:22:12.801418 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.801183 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84a2ff14-a459-43b7-a980-c7aa6ed765fb-installation-pull-secrets\") pod \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " Jul 09 14:22:12.801418 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.801216 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdhh6\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-kube-api-access-fdhh6\") pod \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " Jul 09 14:22:12.801418 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.801245 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-certificates\") pod \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " Jul 09 14:22:12.801418 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.801269 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84a2ff14-a459-43b7-a980-c7aa6ed765fb-trusted-ca\") pod \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " Jul 09 14:22:12.801418 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.801308 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84a2ff14-a459-43b7-a980-c7aa6ed765fb-image-registry-private-configuration\") pod \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\" (UID: \"84a2ff14-a459-43b7-a980-c7aa6ed765fb\") " Jul 09 14:22:12.801909 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.801835 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "84a2ff14-a459-43b7-a980-c7aa6ed765fb" (UID: "84a2ff14-a459-43b7-a980-c7aa6ed765fb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:22:12.802295 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.802264 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84a2ff14-a459-43b7-a980-c7aa6ed765fb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "84a2ff14-a459-43b7-a980-c7aa6ed765fb" (UID: "84a2ff14-a459-43b7-a980-c7aa6ed765fb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:22:12.804110 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.803974 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "84a2ff14-a459-43b7-a980-c7aa6ed765fb" (UID: "84a2ff14-a459-43b7-a980-c7aa6ed765fb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:22:12.804366 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.804217 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "84a2ff14-a459-43b7-a980-c7aa6ed765fb" (UID: "84a2ff14-a459-43b7-a980-c7aa6ed765fb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:22:12.804366 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.804340 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84a2ff14-a459-43b7-a980-c7aa6ed765fb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "84a2ff14-a459-43b7-a980-c7aa6ed765fb" (UID: "84a2ff14-a459-43b7-a980-c7aa6ed765fb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:22:12.804482 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.804373 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-kube-api-access-fdhh6" (OuterVolumeSpecName: "kube-api-access-fdhh6") pod "84a2ff14-a459-43b7-a980-c7aa6ed765fb" (UID: "84a2ff14-a459-43b7-a980-c7aa6ed765fb"). InnerVolumeSpecName "kube-api-access-fdhh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:22:12.804957 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.804937 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84a2ff14-a459-43b7-a980-c7aa6ed765fb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "84a2ff14-a459-43b7-a980-c7aa6ed765fb" (UID: "84a2ff14-a459-43b7-a980-c7aa6ed765fb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:22:12.810270 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.810246 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84a2ff14-a459-43b7-a980-c7aa6ed765fb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "84a2ff14-a459-43b7-a980-c7aa6ed765fb" (UID: "84a2ff14-a459-43b7-a980-c7aa6ed765fb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:22:12.901972 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.901944 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdhh6\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-kube-api-access-fdhh6\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:22:12.901972 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.901971 2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-certificates\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:22:12.902114 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.901986 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84a2ff14-a459-43b7-a980-c7aa6ed765fb-trusted-ca\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:22:12.902114 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.901999 2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84a2ff14-a459-43b7-a980-c7aa6ed765fb-image-registry-private-configuration\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:22:12.902114 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.902011 2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-registry-tls\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:22:12.902114 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.902023 2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84a2ff14-a459-43b7-a980-c7aa6ed765fb-bound-sa-token\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:22:12.902114 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.902034 2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84a2ff14-a459-43b7-a980-c7aa6ed765fb-ca-trust-extracted\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:22:12.902114 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:12.902047 2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84a2ff14-a459-43b7-a980-c7aa6ed765fb-installation-pull-secrets\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:22:13.459428 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:13.459397 2582 generic.go:356] "Generic (PLEG): container finished" podID="84a2ff14-a459-43b7-a980-c7aa6ed765fb" containerID="ad9feb2274abc872f94da3c1ad140ee326df9dbdc27c0055d441e8a23e513132" exitCode=0 Jul 09 14:22:13.459629 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:13.459439 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7649c48849-mdnn4" Jul 09 14:22:13.459629 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:13.459446 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7649c48849-mdnn4" event={"ID":"84a2ff14-a459-43b7-a980-c7aa6ed765fb","Type":"ContainerDied","Data":"ad9feb2274abc872f94da3c1ad140ee326df9dbdc27c0055d441e8a23e513132"} Jul 09 14:22:13.459629 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:13.459486 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7649c48849-mdnn4" event={"ID":"84a2ff14-a459-43b7-a980-c7aa6ed765fb","Type":"ContainerDied","Data":"527c5be66a43e37c421ce6e3bc9d4c19c96d45fd36a7d7bf0b2ce91f674f0723"} Jul 09 14:22:13.459629 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:13.459506 2582 scope.go:117] "RemoveContainer" containerID="ad9feb2274abc872f94da3c1ad140ee326df9dbdc27c0055d441e8a23e513132" Jul 09 14:22:13.468550 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:13.468443 2582 scope.go:117] "RemoveContainer" containerID="ad9feb2274abc872f94da3c1ad140ee326df9dbdc27c0055d441e8a23e513132" Jul 09 14:22:13.468774 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:22:13.468741 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad9feb2274abc872f94da3c1ad140ee326df9dbdc27c0055d441e8a23e513132\": container with ID starting with ad9feb2274abc872f94da3c1ad140ee326df9dbdc27c0055d441e8a23e513132 not found: ID does not exist" containerID="ad9feb2274abc872f94da3c1ad140ee326df9dbdc27c0055d441e8a23e513132" Jul 09 14:22:13.468812 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:13.468771 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad9feb2274abc872f94da3c1ad140ee326df9dbdc27c0055d441e8a23e513132"} err="failed to get container status \"ad9feb2274abc872f94da3c1ad140ee326df9dbdc27c0055d441e8a23e513132\": rpc error: code = NotFound desc = could not find container \"ad9feb2274abc872f94da3c1ad140ee326df9dbdc27c0055d441e8a23e513132\": container with ID starting with ad9feb2274abc872f94da3c1ad140ee326df9dbdc27c0055d441e8a23e513132 not found: ID does not exist" Jul 09 14:22:13.479268 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:13.479247 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7649c48849-mdnn4"] Jul 09 14:22:13.481100 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:13.481083 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7649c48849-mdnn4"] Jul 09 14:22:14.039660 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:14.037911 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84a2ff14-a459-43b7-a980-c7aa6ed765fb" path="/var/lib/kubelet/pods/84a2ff14-a459-43b7-a980-c7aa6ed765fb/volumes" Jul 09 14:22:35.421576 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:22:35.421546 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kjc84" Jul 09 14:25:49.931284 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:25:49.931250 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 14:27:14.494367 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.494327 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r"] Jul 09 14:27:14.495020 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.494640 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84a2ff14-a459-43b7-a980-c7aa6ed765fb" containerName="registry" Jul 09 14:27:14.495020 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.494658 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="84a2ff14-a459-43b7-a980-c7aa6ed765fb" containerName="registry" Jul 09 14:27:14.495020 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.494736 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="84a2ff14-a459-43b7-a980-c7aa6ed765fb" containerName="registry" Jul 09 14:27:14.497595 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.497574 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.500514 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.500499 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 14:27:14.500768 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.500753 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 14:27:14.501269 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.501255 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fdgwt\"" Jul 09 14:27:14.501327 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.501257 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 14:27:14.501684 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.501668 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 14:27:14.519321 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.519298 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r"] Jul 09 14:27:14.630930 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.630891 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.631102 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.630950 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.631102 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.630972 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq2w8\" (UniqueName: \"kubernetes.io/projected/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-kube-api-access-zq2w8\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.631102 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.631026 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-manifests\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.631102 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.631043 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-tmp\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.732235 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.732194 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.732235 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.732234 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq2w8\" (UniqueName: \"kubernetes.io/projected/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-kube-api-access-zq2w8\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.732539 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.732266 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-manifests\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.732539 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.732281 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-tmp\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.732539 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.732305 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.732759 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.732735 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-manifests\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.734583 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.734549 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-tmp\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.734841 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.734820 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.734934 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.734849 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.741135 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.741117 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq2w8\" (UniqueName: \"kubernetes.io/projected/3568bfcb-c3d3-46ff-bb99-7d975be2d2b7-kube-api-access-zq2w8\") pod \"opendatahub-operator-controller-manager-c7757d956-5t57r\" (UID: \"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7\") " pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.808101 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.808074 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:14.986326 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.986283 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r"] Jul 09 14:27:14.989510 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:27:14.989481 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3568bfcb_c3d3_46ff_bb99_7d975be2d2b7.slice/crio-450748afef4366594dd95d65a7ba5d10ddd08989e6f17e3cf9fcbae623c74102 WatchSource:0}: Error finding container 450748afef4366594dd95d65a7ba5d10ddd08989e6f17e3cf9fcbae623c74102: Status 404 returned error can't find the container with id 450748afef4366594dd95d65a7ba5d10ddd08989e6f17e3cf9fcbae623c74102 Jul 09 14:27:14.991299 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:14.991280 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 14:27:15.438787 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:15.438748 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" event={"ID":"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7","Type":"ContainerStarted","Data":"450748afef4366594dd95d65a7ba5d10ddd08989e6f17e3cf9fcbae623c74102"} Jul 09 14:27:18.451165 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:18.451090 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" event={"ID":"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7","Type":"ContainerStarted","Data":"dfa3dfaaa0e8bf5a0c59667f2156459580ffded7777f5cb008aba0b0ac5dab1c"} Jul 09 14:27:19.456484 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:19.456448 2582 generic.go:356] "Generic (PLEG): container finished" podID="3568bfcb-c3d3-46ff-bb99-7d975be2d2b7" containerID="dfa3dfaaa0e8bf5a0c59667f2156459580ffded7777f5cb008aba0b0ac5dab1c" exitCode=0 Jul 09 14:27:19.456844 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:19.456525 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" event={"ID":"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7","Type":"ContainerDied","Data":"dfa3dfaaa0e8bf5a0c59667f2156459580ffded7777f5cb008aba0b0ac5dab1c"} Jul 09 14:27:20.462748 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:20.462701 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" event={"ID":"3568bfcb-c3d3-46ff-bb99-7d975be2d2b7","Type":"ContainerStarted","Data":"2d3617728c476ed93987faed1294b85c46c99506c548021ded60b7436d5abbed"} Jul 09 14:27:20.463273 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:20.462835 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:20.492676 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:20.492629 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" podStartSLOduration=1.798434398 podStartE2EDuration="6.492614646s" totalImagesPullingTime="3.646297184s" totalInitContainerRuntime="1.047883064s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:27:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:27:20.492312178 +0000 UTC m=+390.858406563" watchObservedRunningTime="2026-07-09 14:27:20.492614646 +0000 UTC m=+390.858709057" Jul 09 14:27:31.468343 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:31.468312 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c7757d956-5t57r" Jul 09 14:27:34.755090 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.755058 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77975789f4-69c6k"] Jul 09 14:27:34.767245 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.767218 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77975789f4-69c6k" Jul 09 14:27:34.773387 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.773353 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 14:27:34.773614 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.773593 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 14:27:34.773788 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.773772 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 14:27:34.773978 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.773959 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-r9h78\"" Jul 09 14:27:34.775984 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.775936 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77975789f4-69c6k"] Jul 09 14:27:34.776128 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.776099 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 14:27:34.786582 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.786552 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa562d9c-ec3c-40b5-925f-a6862eab8006-tls-certs\") pod \"kube-auth-proxy-77975789f4-69c6k\" (UID: \"fa562d9c-ec3c-40b5-925f-a6862eab8006\") " pod="openshift-ingress/kube-auth-proxy-77975789f4-69c6k" Jul 09 14:27:34.786738 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.786615 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa562d9c-ec3c-40b5-925f-a6862eab8006-tmp\") pod \"kube-auth-proxy-77975789f4-69c6k\" (UID: \"fa562d9c-ec3c-40b5-925f-a6862eab8006\") " pod="openshift-ingress/kube-auth-proxy-77975789f4-69c6k" Jul 09 14:27:34.786738 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.786658 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzv6f\" (UniqueName: \"kubernetes.io/projected/fa562d9c-ec3c-40b5-925f-a6862eab8006-kube-api-access-jzv6f\") pod \"kube-auth-proxy-77975789f4-69c6k\" (UID: \"fa562d9c-ec3c-40b5-925f-a6862eab8006\") " pod="openshift-ingress/kube-auth-proxy-77975789f4-69c6k" Jul 09 14:27:34.887547 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.887507 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzv6f\" (UniqueName: \"kubernetes.io/projected/fa562d9c-ec3c-40b5-925f-a6862eab8006-kube-api-access-jzv6f\") pod \"kube-auth-proxy-77975789f4-69c6k\" (UID: \"fa562d9c-ec3c-40b5-925f-a6862eab8006\") " pod="openshift-ingress/kube-auth-proxy-77975789f4-69c6k" Jul 09 14:27:34.887729 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.887564 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa562d9c-ec3c-40b5-925f-a6862eab8006-tls-certs\") pod \"kube-auth-proxy-77975789f4-69c6k\" (UID: \"fa562d9c-ec3c-40b5-925f-a6862eab8006\") " pod="openshift-ingress/kube-auth-proxy-77975789f4-69c6k" Jul 09 14:27:34.887729 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.887594 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa562d9c-ec3c-40b5-925f-a6862eab8006-tmp\") pod \"kube-auth-proxy-77975789f4-69c6k\" (UID: \"fa562d9c-ec3c-40b5-925f-a6862eab8006\") " pod="openshift-ingress/kube-auth-proxy-77975789f4-69c6k" Jul 09 14:27:34.889908 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.889860 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa562d9c-ec3c-40b5-925f-a6862eab8006-tmp\") pod \"kube-auth-proxy-77975789f4-69c6k\" (UID: \"fa562d9c-ec3c-40b5-925f-a6862eab8006\") " pod="openshift-ingress/kube-auth-proxy-77975789f4-69c6k" Jul 09 14:27:34.890104 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.890086 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa562d9c-ec3c-40b5-925f-a6862eab8006-tls-certs\") pod \"kube-auth-proxy-77975789f4-69c6k\" (UID: \"fa562d9c-ec3c-40b5-925f-a6862eab8006\") " pod="openshift-ingress/kube-auth-proxy-77975789f4-69c6k" Jul 09 14:27:34.902064 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.902040 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzv6f\" (UniqueName: \"kubernetes.io/projected/fa562d9c-ec3c-40b5-925f-a6862eab8006-kube-api-access-jzv6f\") pod \"kube-auth-proxy-77975789f4-69c6k\" (UID: \"fa562d9c-ec3c-40b5-925f-a6862eab8006\") " pod="openshift-ingress/kube-auth-proxy-77975789f4-69c6k" Jul 09 14:27:34.930935 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.930910 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-95rmq"] Jul 09 14:27:34.934051 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.934036 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-95rmq" Jul 09 14:27:34.936743 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.936726 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-jf2qp\"" Jul 09 14:27:34.936887 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.936726 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 09 14:27:34.944106 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.944085 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-95rmq"] Jul 09 14:27:34.987833 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.987802 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4536ea27-ba7e-41f2-acbd-d9dac53ce64c-cert\") pod \"odh-model-controller-7998c67dc4-95rmq\" (UID: \"4536ea27-ba7e-41f2-acbd-d9dac53ce64c\") " pod="opendatahub/odh-model-controller-7998c67dc4-95rmq" Jul 09 14:27:34.988001 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:34.987837 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz2rl\" (UniqueName: \"kubernetes.io/projected/4536ea27-ba7e-41f2-acbd-d9dac53ce64c-kube-api-access-jz2rl\") pod \"odh-model-controller-7998c67dc4-95rmq\" (UID: \"4536ea27-ba7e-41f2-acbd-d9dac53ce64c\") " pod="opendatahub/odh-model-controller-7998c67dc4-95rmq" Jul 09 14:27:35.082717 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.082684 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77975789f4-69c6k" Jul 09 14:27:35.089564 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.089539 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4536ea27-ba7e-41f2-acbd-d9dac53ce64c-cert\") pod \"odh-model-controller-7998c67dc4-95rmq\" (UID: \"4536ea27-ba7e-41f2-acbd-d9dac53ce64c\") " pod="opendatahub/odh-model-controller-7998c67dc4-95rmq" Jul 09 14:27:35.089713 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.089691 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz2rl\" (UniqueName: \"kubernetes.io/projected/4536ea27-ba7e-41f2-acbd-d9dac53ce64c-kube-api-access-jz2rl\") pod \"odh-model-controller-7998c67dc4-95rmq\" (UID: \"4536ea27-ba7e-41f2-acbd-d9dac53ce64c\") " pod="opendatahub/odh-model-controller-7998c67dc4-95rmq" Jul 09 14:27:35.089807 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:27:35.089611 2582 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 09 14:27:35.089807 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:27:35.089778 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4536ea27-ba7e-41f2-acbd-d9dac53ce64c-cert podName:4536ea27-ba7e-41f2-acbd-d9dac53ce64c nodeName:}" failed. No retries permitted until 2026-07-09 14:27:35.589757513 +0000 UTC m=+405.955851878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4536ea27-ba7e-41f2-acbd-d9dac53ce64c-cert") pod "odh-model-controller-7998c67dc4-95rmq" (UID: "4536ea27-ba7e-41f2-acbd-d9dac53ce64c") : secret "odh-model-controller-webhook-cert" not found Jul 09 14:27:35.098793 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.098772 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz2rl\" (UniqueName: \"kubernetes.io/projected/4536ea27-ba7e-41f2-acbd-d9dac53ce64c-kube-api-access-jz2rl\") pod \"odh-model-controller-7998c67dc4-95rmq\" (UID: \"4536ea27-ba7e-41f2-acbd-d9dac53ce64c\") " pod="opendatahub/odh-model-controller-7998c67dc4-95rmq" Jul 09 14:27:35.245822 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.245682 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77975789f4-69c6k"] Jul 09 14:27:35.248503 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:27:35.248470 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa562d9c_ec3c_40b5_925f_a6862eab8006.slice/crio-0aff09cba7d9e054d1cb5ab8f83dc0588e5f09092375b14d101dd189126df8fd WatchSource:0}: Error finding container 0aff09cba7d9e054d1cb5ab8f83dc0588e5f09092375b14d101dd189126df8fd: Status 404 returned error can't find the container with id 0aff09cba7d9e054d1cb5ab8f83dc0588e5f09092375b14d101dd189126df8fd Jul 09 14:27:35.514053 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.513974 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77975789f4-69c6k" event={"ID":"fa562d9c-ec3c-40b5-925f-a6862eab8006","Type":"ContainerStarted","Data":"0aff09cba7d9e054d1cb5ab8f83dc0588e5f09092375b14d101dd189126df8fd"} Jul 09 14:27:35.593122 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.593086 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4536ea27-ba7e-41f2-acbd-d9dac53ce64c-cert\") pod \"odh-model-controller-7998c67dc4-95rmq\" (UID: \"4536ea27-ba7e-41f2-acbd-d9dac53ce64c\") " pod="opendatahub/odh-model-controller-7998c67dc4-95rmq" Jul 09 14:27:35.595740 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.595707 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4536ea27-ba7e-41f2-acbd-d9dac53ce64c-cert\") pod \"odh-model-controller-7998c67dc4-95rmq\" (UID: \"4536ea27-ba7e-41f2-acbd-d9dac53ce64c\") " pod="opendatahub/odh-model-controller-7998c67dc4-95rmq" Jul 09 14:27:35.838632 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.838592 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4fwdr"] Jul 09 14:27:35.843373 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.843352 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4fwdr" Jul 09 14:27:35.846525 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.846498 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-l4zck\"" Jul 09 14:27:35.846656 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.846590 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 09 14:27:35.846799 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.846781 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-95rmq" Jul 09 14:27:35.853126 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.852436 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4fwdr"] Jul 09 14:27:35.895884 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.895834 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9jgc\" (UniqueName: \"kubernetes.io/projected/59035872-ab43-47f9-bcfe-5e1bbb034078-kube-api-access-w9jgc\") pod \"model-serving-api-54d89bd5fc-4fwdr\" (UID: \"59035872-ab43-47f9-bcfe-5e1bbb034078\") " pod="opendatahub/model-serving-api-54d89bd5fc-4fwdr" Jul 09 14:27:35.896056 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.895961 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59035872-ab43-47f9-bcfe-5e1bbb034078-tls-certs\") pod \"model-serving-api-54d89bd5fc-4fwdr\" (UID: \"59035872-ab43-47f9-bcfe-5e1bbb034078\") " pod="opendatahub/model-serving-api-54d89bd5fc-4fwdr" Jul 09 14:27:35.996816 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.996760 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59035872-ab43-47f9-bcfe-5e1bbb034078-tls-certs\") pod \"model-serving-api-54d89bd5fc-4fwdr\" (UID: \"59035872-ab43-47f9-bcfe-5e1bbb034078\") " pod="opendatahub/model-serving-api-54d89bd5fc-4fwdr" Jul 09 14:27:35.997037 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:35.996825 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9jgc\" (UniqueName: \"kubernetes.io/projected/59035872-ab43-47f9-bcfe-5e1bbb034078-kube-api-access-w9jgc\") pod \"model-serving-api-54d89bd5fc-4fwdr\" (UID: \"59035872-ab43-47f9-bcfe-5e1bbb034078\") " pod="opendatahub/model-serving-api-54d89bd5fc-4fwdr" Jul 09 14:27:35.997387 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:27:35.997224 2582 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 09 14:27:35.997387 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:27:35.997291 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59035872-ab43-47f9-bcfe-5e1bbb034078-tls-certs podName:59035872-ab43-47f9-bcfe-5e1bbb034078 nodeName:}" failed. No retries permitted until 2026-07-09 14:27:36.497270604 +0000 UTC m=+406.863364968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/59035872-ab43-47f9-bcfe-5e1bbb034078-tls-certs") pod "model-serving-api-54d89bd5fc-4fwdr" (UID: "59035872-ab43-47f9-bcfe-5e1bbb034078") : secret "model-serving-api-tls" not found Jul 09 14:27:36.004618 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.004503 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-95rmq"] Jul 09 14:27:36.007921 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:27:36.007831 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4536ea27_ba7e_41f2_acbd_d9dac53ce64c.slice/crio-fd1e186a7484c256c3402ef4ec225276c8a1e149ff9821dde2526de4af68ae4d WatchSource:0}: Error finding container fd1e186a7484c256c3402ef4ec225276c8a1e149ff9821dde2526de4af68ae4d: Status 404 returned error can't find the container with id fd1e186a7484c256c3402ef4ec225276c8a1e149ff9821dde2526de4af68ae4d Jul 09 14:27:36.013384 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.013359 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9jgc\" (UniqueName: \"kubernetes.io/projected/59035872-ab43-47f9-bcfe-5e1bbb034078-kube-api-access-w9jgc\") pod \"model-serving-api-54d89bd5fc-4fwdr\" (UID: \"59035872-ab43-47f9-bcfe-5e1bbb034078\") " pod="opendatahub/model-serving-api-54d89bd5fc-4fwdr" Jul 09 14:27:36.056271 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.056234 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm"] Jul 09 14:27:36.059953 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.059927 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.062928 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.062531 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 14:27:36.062928 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.062544 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 14:27:36.062928 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.062550 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 14:27:36.062928 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.062594 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8tbbg\"" Jul 09 14:27:36.062928 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.062531 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 14:27:36.073964 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.073932 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm"] Jul 09 14:27:36.098174 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.098046 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.098174 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.098112 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq5km\" (UniqueName: \"kubernetes.io/projected/deb8a62a-84e9-480c-9323-2e1e6c03b72c-kube-api-access-sq5km\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.098403 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.098205 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-token\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.098403 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.098285 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/deb8a62a-84e9-480c-9323-2e1e6c03b72c-local-certs\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.098403 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.098375 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.098600 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.098408 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.098600 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.098434 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-cacerts\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.198848 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.198809 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.198848 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.198856 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq5km\" (UniqueName: \"kubernetes.io/projected/deb8a62a-84e9-480c-9323-2e1e6c03b72c-kube-api-access-sq5km\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.199100 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.198902 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-token\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.199100 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.198954 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/deb8a62a-84e9-480c-9323-2e1e6c03b72c-local-certs\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.199100 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.199017 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.199100 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.199045 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.199100 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.199069 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-cacerts\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.199576 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.199516 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.202115 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.202089 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.202229 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.202110 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-cacerts\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.202229 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.202129 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/deb8a62a-84e9-480c-9323-2e1e6c03b72c-local-certs\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.202229 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.202144 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.211464 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.211414 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq5km\" (UniqueName: \"kubernetes.io/projected/deb8a62a-84e9-480c-9323-2e1e6c03b72c-kube-api-access-sq5km\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.212499 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.212472 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-token\") pod \"istiod-openshift-gateway-7574459656-nbmtm\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.372234 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.372145 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:36.501368 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.501338 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59035872-ab43-47f9-bcfe-5e1bbb034078-tls-certs\") pod \"model-serving-api-54d89bd5fc-4fwdr\" (UID: \"59035872-ab43-47f9-bcfe-5e1bbb034078\") " pod="opendatahub/model-serving-api-54d89bd5fc-4fwdr" Jul 09 14:27:36.503922 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.503865 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59035872-ab43-47f9-bcfe-5e1bbb034078-tls-certs\") pod \"model-serving-api-54d89bd5fc-4fwdr\" (UID: \"59035872-ab43-47f9-bcfe-5e1bbb034078\") " pod="opendatahub/model-serving-api-54d89bd5fc-4fwdr" Jul 09 14:27:36.518693 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.518657 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-95rmq" event={"ID":"4536ea27-ba7e-41f2-acbd-d9dac53ce64c","Type":"ContainerStarted","Data":"fd1e186a7484c256c3402ef4ec225276c8a1e149ff9821dde2526de4af68ae4d"} Jul 09 14:27:36.527896 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.527840 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm"] Jul 09 14:27:36.530466 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:27:36.530441 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeb8a62a_84e9_480c_9323_2e1e6c03b72c.slice/crio-027d5a1bbe431b6080703495cc123a3ac927a0b1e4a4d2ce6651000858798e01 WatchSource:0}: Error finding container 027d5a1bbe431b6080703495cc123a3ac927a0b1e4a4d2ce6651000858798e01: Status 404 returned error can't find the container with id 027d5a1bbe431b6080703495cc123a3ac927a0b1e4a4d2ce6651000858798e01 Jul 09 14:27:36.757661 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.757364 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4fwdr" Jul 09 14:27:36.931292 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:36.931240 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4fwdr"] Jul 09 14:27:36.936133 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:27:36.936104 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59035872_ab43_47f9_bcfe_5e1bbb034078.slice/crio-3389035993f369ab08a1e01cf20ed7dad028b865812d6323551c263d4e0d165e WatchSource:0}: Error finding container 3389035993f369ab08a1e01cf20ed7dad028b865812d6323551c263d4e0d165e: Status 404 returned error can't find the container with id 3389035993f369ab08a1e01cf20ed7dad028b865812d6323551c263d4e0d165e Jul 09 14:27:37.530092 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:37.530045 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4fwdr" event={"ID":"59035872-ab43-47f9-bcfe-5e1bbb034078","Type":"ContainerStarted","Data":"3389035993f369ab08a1e01cf20ed7dad028b865812d6323551c263d4e0d165e"} Jul 09 14:27:37.533973 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:37.533936 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" event={"ID":"deb8a62a-84e9-480c-9323-2e1e6c03b72c","Type":"ContainerStarted","Data":"027d5a1bbe431b6080703495cc123a3ac927a0b1e4a4d2ce6651000858798e01"} Jul 09 14:27:41.355710 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:41.355670 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 09 14:27:41.356097 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:41.355758 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 09 14:27:41.554245 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:41.554174 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77975789f4-69c6k" event={"ID":"fa562d9c-ec3c-40b5-925f-a6862eab8006","Type":"ContainerStarted","Data":"b8758a525b781bd799f17ae4da1cb8c4ea9299c893325e6f289e1ca23add8b11"} Jul 09 14:27:41.556203 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:41.556173 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-95rmq" event={"ID":"4536ea27-ba7e-41f2-acbd-d9dac53ce64c","Type":"ContainerStarted","Data":"36c1d16ad2267cfed52c9cc4d35840cfde8a51ce34a1df4a79da65759acd9f66"} Jul 09 14:27:41.556343 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:41.556301 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-95rmq" Jul 09 14:27:41.557928 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:41.557901 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4fwdr" event={"ID":"59035872-ab43-47f9-bcfe-5e1bbb034078","Type":"ContainerStarted","Data":"9941ed09ddcef8af95d08ef447906e2b924994cc9258a1e3d1c6bdbb95ea54e7"} Jul 09 14:27:41.558009 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:41.557958 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-4fwdr" Jul 09 14:27:41.560292 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:41.560254 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" event={"ID":"deb8a62a-84e9-480c-9323-2e1e6c03b72c","Type":"ContainerStarted","Data":"2ba55df4f00718b542ad74d27c8e10918411263e4c65bdf549f22e291fac0fc8"} Jul 09 14:27:41.560571 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:41.560378 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:41.572311 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:41.572142 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77975789f4-69c6k" podStartSLOduration=1.472081615 podStartE2EDuration="7.572125766s" totalImagesPullingTime="6.100044151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:27:41.571845434 +0000 UTC m=+411.937939831" watchObservedRunningTime="2026-07-09 14:27:41.572125766 +0000 UTC m=+411.938220153" Jul 09 14:27:41.591079 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:41.591023 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-95rmq" podStartSLOduration=2.252183015 podStartE2EDuration="7.591001273s" totalImagesPullingTime="5.338818258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:27:41.588918806 +0000 UTC m=+411.955013193" watchObservedRunningTime="2026-07-09 14:27:41.591001273 +0000 UTC m=+411.957095660" Jul 09 14:27:41.615078 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:41.615013 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-4fwdr" podStartSLOduration=2.2042953499999998 podStartE2EDuration="6.614992217s" totalImagesPullingTime="4.410696867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:27:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:27:41.612646356 +0000 UTC m=+411.978740742" watchObservedRunningTime="2026-07-09 14:27:41.614992217 +0000 UTC m=+411.981086606" Jul 09 14:27:42.568229 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:42.568194 2582 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-nbmtm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 14:27:42.568664 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:42.568253 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" podUID="deb8a62a-84e9-480c-9323-2e1e6c03b72c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:27:45.566555 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:45.566525 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:45.591020 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:45.590969 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" podStartSLOduration=4.767811371 podStartE2EDuration="9.590952671s" totalImagesPullingTime="4.8231413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:27:41.641417159 +0000 UTC m=+412.007511548" watchObservedRunningTime="2026-07-09 14:27:45.590952671 +0000 UTC m=+415.957047057" Jul 09 14:27:46.693255 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.693217 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5"] Jul 09 14:27:46.697256 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.697230 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5" Jul 09 14:27:46.701359 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.701337 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 09 14:27:46.701658 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.701606 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-47gjt\"" Jul 09 14:27:46.715275 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.715251 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt"] Jul 09 14:27:46.718719 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.718695 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt" Jul 09 14:27:46.721617 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.721434 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-rv27n\"" Jul 09 14:27:46.722353 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.722334 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 09 14:27:46.723700 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.723647 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5"] Jul 09 14:27:46.747321 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.747263 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt"] Jul 09 14:27:46.781599 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.781567 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/980c62dc-ce17-4ac3-b661-e9a4faefdbc2-cert\") pod \"kserve-controller-manager-d6f4b65b8-tpgz5\" (UID: \"980c62dc-ce17-4ac3-b661-e9a4faefdbc2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5" Jul 09 14:27:46.781770 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.781605 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqrsq\" (UniqueName: \"kubernetes.io/projected/0e7819cf-6cc4-4597-8e34-b5fb8a86101a-kube-api-access-rqrsq\") pod \"llmisvc-controller-manager-777c6bd7d5-95czt\" (UID: \"0e7819cf-6cc4-4597-8e34-b5fb8a86101a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt" Jul 09 14:27:46.781770 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.781671 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e7819cf-6cc4-4597-8e34-b5fb8a86101a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-95czt\" (UID: \"0e7819cf-6cc4-4597-8e34-b5fb8a86101a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt" Jul 09 14:27:46.781770 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.781754 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbw7v\" (UniqueName: \"kubernetes.io/projected/980c62dc-ce17-4ac3-b661-e9a4faefdbc2-kube-api-access-lbw7v\") pod \"kserve-controller-manager-d6f4b65b8-tpgz5\" (UID: \"980c62dc-ce17-4ac3-b661-e9a4faefdbc2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5" Jul 09 14:27:46.882649 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.882613 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/980c62dc-ce17-4ac3-b661-e9a4faefdbc2-cert\") pod \"kserve-controller-manager-d6f4b65b8-tpgz5\" (UID: \"980c62dc-ce17-4ac3-b661-e9a4faefdbc2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5" Jul 09 14:27:46.882649 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.882652 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqrsq\" (UniqueName: \"kubernetes.io/projected/0e7819cf-6cc4-4597-8e34-b5fb8a86101a-kube-api-access-rqrsq\") pod \"llmisvc-controller-manager-777c6bd7d5-95czt\" (UID: \"0e7819cf-6cc4-4597-8e34-b5fb8a86101a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt" Jul 09 14:27:46.882917 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.882688 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e7819cf-6cc4-4597-8e34-b5fb8a86101a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-95czt\" (UID: \"0e7819cf-6cc4-4597-8e34-b5fb8a86101a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt" Jul 09 14:27:46.882917 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.882727 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbw7v\" (UniqueName: \"kubernetes.io/projected/980c62dc-ce17-4ac3-b661-e9a4faefdbc2-kube-api-access-lbw7v\") pod \"kserve-controller-manager-d6f4b65b8-tpgz5\" (UID: \"980c62dc-ce17-4ac3-b661-e9a4faefdbc2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5" Jul 09 14:27:46.882917 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:27:46.882850 2582 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 09 14:27:46.883076 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:27:46.882957 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7819cf-6cc4-4597-8e34-b5fb8a86101a-cert podName:0e7819cf-6cc4-4597-8e34-b5fb8a86101a nodeName:}" failed. No retries permitted until 2026-07-09 14:27:47.382938884 +0000 UTC m=+417.749033252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0e7819cf-6cc4-4597-8e34-b5fb8a86101a-cert") pod "llmisvc-controller-manager-777c6bd7d5-95czt" (UID: "0e7819cf-6cc4-4597-8e34-b5fb8a86101a") : secret "llmisvc-webhook-server-cert" not found Jul 09 14:27:46.885319 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.885298 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/980c62dc-ce17-4ac3-b661-e9a4faefdbc2-cert\") pod \"kserve-controller-manager-d6f4b65b8-tpgz5\" (UID: \"980c62dc-ce17-4ac3-b661-e9a4faefdbc2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5" Jul 09 14:27:46.901492 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.901469 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqrsq\" (UniqueName: \"kubernetes.io/projected/0e7819cf-6cc4-4597-8e34-b5fb8a86101a-kube-api-access-rqrsq\") pod \"llmisvc-controller-manager-777c6bd7d5-95czt\" (UID: \"0e7819cf-6cc4-4597-8e34-b5fb8a86101a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt" Jul 09 14:27:46.901614 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:46.901556 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbw7v\" (UniqueName: \"kubernetes.io/projected/980c62dc-ce17-4ac3-b661-e9a4faefdbc2-kube-api-access-lbw7v\") pod \"kserve-controller-manager-d6f4b65b8-tpgz5\" (UID: \"980c62dc-ce17-4ac3-b661-e9a4faefdbc2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5" Jul 09 14:27:47.011084 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:47.010997 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5" Jul 09 14:27:47.159321 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:47.159288 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5"] Jul 09 14:27:47.164831 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:27:47.164803 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod980c62dc_ce17_4ac3_b661_e9a4faefdbc2.slice/crio-a1a81e82f64d792e37efca1ee161e0f9ccc6332b9ba7924a75cd4c1a84838e40 WatchSource:0}: Error finding container a1a81e82f64d792e37efca1ee161e0f9ccc6332b9ba7924a75cd4c1a84838e40: Status 404 returned error can't find the container with id a1a81e82f64d792e37efca1ee161e0f9ccc6332b9ba7924a75cd4c1a84838e40 Jul 09 14:27:47.387249 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:47.387212 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e7819cf-6cc4-4597-8e34-b5fb8a86101a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-95czt\" (UID: \"0e7819cf-6cc4-4597-8e34-b5fb8a86101a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt" Jul 09 14:27:47.389803 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:47.389780 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e7819cf-6cc4-4597-8e34-b5fb8a86101a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-95czt\" (UID: \"0e7819cf-6cc4-4597-8e34-b5fb8a86101a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt" Jul 09 14:27:47.583992 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:47.583954 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5" event={"ID":"980c62dc-ce17-4ac3-b661-e9a4faefdbc2","Type":"ContainerStarted","Data":"a1a81e82f64d792e37efca1ee161e0f9ccc6332b9ba7924a75cd4c1a84838e40"} Jul 09 14:27:47.634323 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:47.634292 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt" Jul 09 14:27:47.767558 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:47.767533 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt"] Jul 09 14:27:47.770291 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:27:47.770248 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0e7819cf_6cc4_4597_8e34_b5fb8a86101a.slice/crio-7ed40bf0120475413ded533ce08a84a44af1f480618e181fe3b4dd4cf8ffa199 WatchSource:0}: Error finding container 7ed40bf0120475413ded533ce08a84a44af1f480618e181fe3b4dd4cf8ffa199: Status 404 returned error can't find the container with id 7ed40bf0120475413ded533ce08a84a44af1f480618e181fe3b4dd4cf8ffa199 Jul 09 14:27:48.589345 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:48.589307 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt" event={"ID":"0e7819cf-6cc4-4597-8e34-b5fb8a86101a","Type":"ContainerStarted","Data":"7ed40bf0120475413ded533ce08a84a44af1f480618e181fe3b4dd4cf8ffa199"} Jul 09 14:27:49.641633 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:49.641597 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm"] Jul 09 14:27:49.642110 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:49.641841 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" podUID="deb8a62a-84e9-480c-9323-2e1e6c03b72c" containerName="discovery" containerID="cri-o://2ba55df4f00718b542ad74d27c8e10918411263e4c65bdf549f22e291fac0fc8" gracePeriod=30 Jul 09 14:27:49.912959 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:49.912889 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:50.017100 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.017068 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-token\") pod \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " Jul 09 14:27:50.017277 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.017127 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-csr-ca-configmap\") pod \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " Jul 09 14:27:50.017277 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.017154 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/deb8a62a-84e9-480c-9323-2e1e6c03b72c-local-certs\") pod \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " Jul 09 14:27:50.017277 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.017183 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-cacerts\") pod \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " Jul 09 14:27:50.017277 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.017225 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-kubeconfig\") pod \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " Jul 09 14:27:50.017277 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.017271 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-csr-dns-cert\") pod \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " Jul 09 14:27:50.017481 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.017328 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq5km\" (UniqueName: \"kubernetes.io/projected/deb8a62a-84e9-480c-9323-2e1e6c03b72c-kube-api-access-sq5km\") pod \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\" (UID: \"deb8a62a-84e9-480c-9323-2e1e6c03b72c\") " Jul 09 14:27:50.017628 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.017596 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "deb8a62a-84e9-480c-9323-2e1e6c03b72c" (UID: "deb8a62a-84e9-480c-9323-2e1e6c03b72c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:27:50.020127 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.020080 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-token" (OuterVolumeSpecName: "istio-token") pod "deb8a62a-84e9-480c-9323-2e1e6c03b72c" (UID: "deb8a62a-84e9-480c-9323-2e1e6c03b72c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:27:50.020289 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.020263 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-cacerts" (OuterVolumeSpecName: "cacerts") pod "deb8a62a-84e9-480c-9323-2e1e6c03b72c" (UID: "deb8a62a-84e9-480c-9323-2e1e6c03b72c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:27:50.020345 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.020299 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deb8a62a-84e9-480c-9323-2e1e6c03b72c-kube-api-access-sq5km" (OuterVolumeSpecName: "kube-api-access-sq5km") pod "deb8a62a-84e9-480c-9323-2e1e6c03b72c" (UID: "deb8a62a-84e9-480c-9323-2e1e6c03b72c"). InnerVolumeSpecName "kube-api-access-sq5km". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:27:50.020439 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.020420 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "deb8a62a-84e9-480c-9323-2e1e6c03b72c" (UID: "deb8a62a-84e9-480c-9323-2e1e6c03b72c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:27:50.021098 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.021034 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "deb8a62a-84e9-480c-9323-2e1e6c03b72c" (UID: "deb8a62a-84e9-480c-9323-2e1e6c03b72c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:27:50.021205 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.021140 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deb8a62a-84e9-480c-9323-2e1e6c03b72c-local-certs" (OuterVolumeSpecName: "local-certs") pod "deb8a62a-84e9-480c-9323-2e1e6c03b72c" (UID: "deb8a62a-84e9-480c-9323-2e1e6c03b72c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:27:50.118183 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.118155 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-token\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:27:50.118183 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.118180 2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-csr-ca-configmap\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:27:50.118183 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.118190 2582 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/deb8a62a-84e9-480c-9323-2e1e6c03b72c-local-certs\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:27:50.118416 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.118198 2582 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-cacerts\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:27:50.118416 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.118207 2582 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-kubeconfig\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:27:50.118416 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.118214 2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/deb8a62a-84e9-480c-9323-2e1e6c03b72c-istio-csr-dns-cert\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:27:50.118416 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.118221 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq5km\" (UniqueName: \"kubernetes.io/projected/deb8a62a-84e9-480c-9323-2e1e6c03b72c-kube-api-access-sq5km\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:27:50.601889 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.601833 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5" event={"ID":"980c62dc-ce17-4ac3-b661-e9a4faefdbc2","Type":"ContainerStarted","Data":"e0ec3997c35cb9ed8ba4e7358e0af1ee326befd6361eabd57b158f8b63eabbc6"} Jul 09 14:27:50.602080 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.602014 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5" Jul 09 14:27:50.603406 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.603373 2582 generic.go:356] "Generic (PLEG): container finished" podID="deb8a62a-84e9-480c-9323-2e1e6c03b72c" containerID="2ba55df4f00718b542ad74d27c8e10918411263e4c65bdf549f22e291fac0fc8" exitCode=0 Jul 09 14:27:50.603522 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.603421 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" Jul 09 14:27:50.603522 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.603436 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" event={"ID":"deb8a62a-84e9-480c-9323-2e1e6c03b72c","Type":"ContainerDied","Data":"2ba55df4f00718b542ad74d27c8e10918411263e4c65bdf549f22e291fac0fc8"} Jul 09 14:27:50.603522 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.603490 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm" event={"ID":"deb8a62a-84e9-480c-9323-2e1e6c03b72c","Type":"ContainerDied","Data":"027d5a1bbe431b6080703495cc123a3ac927a0b1e4a4d2ce6651000858798e01"} Jul 09 14:27:50.603626 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.603538 2582 scope.go:117] "RemoveContainer" containerID="2ba55df4f00718b542ad74d27c8e10918411263e4c65bdf549f22e291fac0fc8" Jul 09 14:27:50.634652 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.634596 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5" podStartSLOduration=1.9998060579999999 podStartE2EDuration="4.634578519s" totalImagesPullingTime="2.634772461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:27:50.631746502 +0000 UTC m=+420.997840902" watchObservedRunningTime="2026-07-09 14:27:50.634578519 +0000 UTC m=+421.000672910" Jul 09 14:27:50.652895 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.652847 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm"] Jul 09 14:27:50.657317 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.657293 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nbmtm"] Jul 09 14:27:50.759586 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.759435 2582 scope.go:117] "RemoveContainer" containerID="2ba55df4f00718b542ad74d27c8e10918411263e4c65bdf549f22e291fac0fc8" Jul 09 14:27:50.759810 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:27:50.759789 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ba55df4f00718b542ad74d27c8e10918411263e4c65bdf549f22e291fac0fc8\": container with ID starting with 2ba55df4f00718b542ad74d27c8e10918411263e4c65bdf549f22e291fac0fc8 not found: ID does not exist" containerID="2ba55df4f00718b542ad74d27c8e10918411263e4c65bdf549f22e291fac0fc8" Jul 09 14:27:50.759864 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:50.759820 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ba55df4f00718b542ad74d27c8e10918411263e4c65bdf549f22e291fac0fc8"} err="failed to get container status \"2ba55df4f00718b542ad74d27c8e10918411263e4c65bdf549f22e291fac0fc8\": rpc error: code = NotFound desc = could not find container \"2ba55df4f00718b542ad74d27c8e10918411263e4c65bdf549f22e291fac0fc8\": container with ID starting with 2ba55df4f00718b542ad74d27c8e10918411263e4c65bdf549f22e291fac0fc8 not found: ID does not exist" Jul 09 14:27:51.610556 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:51.610514 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt" event={"ID":"0e7819cf-6cc4-4597-8e34-b5fb8a86101a","Type":"ContainerStarted","Data":"2dbcf54e2ac88a95c17b7d8ef312c0427721b183e563e817c2b327926e71fd3e"} Jul 09 14:27:51.634697 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:51.634651 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt" podStartSLOduration=2.600604938 podStartE2EDuration="5.634634792s" totalImagesPullingTime="3.034029854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:27:51.63434888 +0000 UTC m=+422.000443268" watchObservedRunningTime="2026-07-09 14:27:51.634634792 +0000 UTC m=+422.000729169" Jul 09 14:27:52.037407 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:52.037321 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="deb8a62a-84e9-480c-9323-2e1e6c03b72c" path="/var/lib/kubelet/pods/deb8a62a-84e9-480c-9323-2e1e6c03b72c/volumes" Jul 09 14:27:52.569387 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:52.569356 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-95rmq" Jul 09 14:27:52.571429 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:52.571405 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-4fwdr" Jul 09 14:27:52.614773 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:27:52.614738 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt" Jul 09 14:28:21.616404 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:21.616320 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tpgz5" Jul 09 14:28:23.620505 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:23.620469 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-95czt" Jul 09 14:28:35.538178 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.538142 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nkdjd"] Jul 09 14:28:35.538569 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.538444 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="deb8a62a-84e9-480c-9323-2e1e6c03b72c" containerName="discovery" Jul 09 14:28:35.538569 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.538455 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="deb8a62a-84e9-480c-9323-2e1e6c03b72c" containerName="discovery" Jul 09 14:28:35.538569 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.538507 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="deb8a62a-84e9-480c-9323-2e1e6c03b72c" containerName="discovery" Jul 09 14:28:35.549588 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.549556 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nkdjd" Jul 09 14:28:35.550369 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.550339 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nkdjd"] Jul 09 14:28:35.553516 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.553493 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 14:28:35.553667 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.553553 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 14:28:35.554491 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.554451 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-lc6tm\"" Jul 09 14:28:35.585895 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.585851 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzrbm\" (UniqueName: \"kubernetes.io/projected/c51fb5d1-2991-4721-856d-4dae1e573184-kube-api-access-rzrbm\") pod \"kuadrant-operator-catalog-nkdjd\" (UID: \"c51fb5d1-2991-4721-856d-4dae1e573184\") " pod="kuadrant-system/kuadrant-operator-catalog-nkdjd" Jul 09 14:28:35.686476 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.686444 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzrbm\" (UniqueName: \"kubernetes.io/projected/c51fb5d1-2991-4721-856d-4dae1e573184-kube-api-access-rzrbm\") pod \"kuadrant-operator-catalog-nkdjd\" (UID: \"c51fb5d1-2991-4721-856d-4dae1e573184\") " pod="kuadrant-system/kuadrant-operator-catalog-nkdjd" Jul 09 14:28:35.694699 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.694672 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzrbm\" (UniqueName: \"kubernetes.io/projected/c51fb5d1-2991-4721-856d-4dae1e573184-kube-api-access-rzrbm\") pod \"kuadrant-operator-catalog-nkdjd\" (UID: \"c51fb5d1-2991-4721-856d-4dae1e573184\") " pod="kuadrant-system/kuadrant-operator-catalog-nkdjd" Jul 09 14:28:35.860797 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.860759 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nkdjd" Jul 09 14:28:35.899633 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.899599 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nkdjd"] Jul 09 14:28:35.992707 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:35.992676 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nkdjd"] Jul 09 14:28:35.996865 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:28:35.996841 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc51fb5d1_2991_4721_856d_4dae1e573184.slice/crio-71843fb500c031e2f6bf81d97f2d0727228f472e04cf5691143bab0757fddb94 WatchSource:0}: Error finding container 71843fb500c031e2f6bf81d97f2d0727228f472e04cf5691143bab0757fddb94: Status 404 returned error can't find the container with id 71843fb500c031e2f6bf81d97f2d0727228f472e04cf5691143bab0757fddb94 Jul 09 14:28:36.114690 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:36.114612 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pjhd7"] Jul 09 14:28:36.119081 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:36.119063 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pjhd7" Jul 09 14:28:36.129455 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:36.129432 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pjhd7"] Jul 09 14:28:36.190097 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:36.190074 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r5sw\" (UniqueName: \"kubernetes.io/projected/4ca4616b-0e80-4635-a6bc-1cbd8d14602b-kube-api-access-5r5sw\") pod \"kuadrant-operator-catalog-pjhd7\" (UID: \"4ca4616b-0e80-4635-a6bc-1cbd8d14602b\") " pod="kuadrant-system/kuadrant-operator-catalog-pjhd7" Jul 09 14:28:36.291375 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:36.291349 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r5sw\" (UniqueName: \"kubernetes.io/projected/4ca4616b-0e80-4635-a6bc-1cbd8d14602b-kube-api-access-5r5sw\") pod \"kuadrant-operator-catalog-pjhd7\" (UID: \"4ca4616b-0e80-4635-a6bc-1cbd8d14602b\") " pod="kuadrant-system/kuadrant-operator-catalog-pjhd7" Jul 09 14:28:36.300233 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:36.300208 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r5sw\" (UniqueName: \"kubernetes.io/projected/4ca4616b-0e80-4635-a6bc-1cbd8d14602b-kube-api-access-5r5sw\") pod \"kuadrant-operator-catalog-pjhd7\" (UID: \"4ca4616b-0e80-4635-a6bc-1cbd8d14602b\") " pod="kuadrant-system/kuadrant-operator-catalog-pjhd7" Jul 09 14:28:36.428775 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:36.428695 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pjhd7" Jul 09 14:28:36.560421 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:36.560395 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pjhd7"] Jul 09 14:28:36.562852 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:28:36.562828 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca4616b_0e80_4635_a6bc_1cbd8d14602b.slice/crio-706184b084e6f84a2c26dd982f2e64a632a3fdbdbce549b1cbf78551b74ebe73 WatchSource:0}: Error finding container 706184b084e6f84a2c26dd982f2e64a632a3fdbdbce549b1cbf78551b74ebe73: Status 404 returned error can't find the container with id 706184b084e6f84a2c26dd982f2e64a632a3fdbdbce549b1cbf78551b74ebe73 Jul 09 14:28:36.792786 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:36.792709 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nkdjd" event={"ID":"c51fb5d1-2991-4721-856d-4dae1e573184","Type":"ContainerStarted","Data":"71843fb500c031e2f6bf81d97f2d0727228f472e04cf5691143bab0757fddb94"} Jul 09 14:28:36.793925 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:36.793897 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pjhd7" event={"ID":"4ca4616b-0e80-4635-a6bc-1cbd8d14602b","Type":"ContainerStarted","Data":"706184b084e6f84a2c26dd982f2e64a632a3fdbdbce549b1cbf78551b74ebe73"} Jul 09 14:28:38.805627 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:38.805587 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nkdjd" event={"ID":"c51fb5d1-2991-4721-856d-4dae1e573184","Type":"ContainerStarted","Data":"c38b96669d5aa5773c014133eec144334ca5313082ed3b48f182ce1af3d575ed"} Jul 09 14:28:38.805627 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:38.805621 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-nkdjd" podUID="c51fb5d1-2991-4721-856d-4dae1e573184" containerName="registry-server" containerID="cri-o://c38b96669d5aa5773c014133eec144334ca5313082ed3b48f182ce1af3d575ed" gracePeriod=2 Jul 09 14:28:38.807299 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:38.807275 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pjhd7" event={"ID":"4ca4616b-0e80-4635-a6bc-1cbd8d14602b","Type":"ContainerStarted","Data":"1551abf7ad3b032a75e6e4789fc98b249448422e78fbc1b8c4b0b95a960f3474"} Jul 09 14:28:38.821680 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:38.821640 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-nkdjd" podStartSLOduration=1.7329255 podStartE2EDuration="3.821626698s" totalImagesPullingTime="2.088701198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:28:38.820894776 +0000 UTC m=+469.186989154" watchObservedRunningTime="2026-07-09 14:28:38.821626698 +0000 UTC m=+469.187721083" Jul 09 14:28:38.837207 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:38.837165 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pjhd7" podStartSLOduration=1.313668185 podStartE2EDuration="2.837150504s" totalImagesPullingTime="1.523482319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:28:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:28:38.836492074 +0000 UTC m=+469.202586461" watchObservedRunningTime="2026-07-09 14:28:38.837150504 +0000 UTC m=+469.203244904" Jul 09 14:28:39.052241 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:39.052218 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nkdjd" Jul 09 14:28:39.114858 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:39.114825 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzrbm\" (UniqueName: \"kubernetes.io/projected/c51fb5d1-2991-4721-856d-4dae1e573184-kube-api-access-rzrbm\") pod \"c51fb5d1-2991-4721-856d-4dae1e573184\" (UID: \"c51fb5d1-2991-4721-856d-4dae1e573184\") " Jul 09 14:28:39.117074 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:39.117048 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c51fb5d1-2991-4721-856d-4dae1e573184-kube-api-access-rzrbm" (OuterVolumeSpecName: "kube-api-access-rzrbm") pod "c51fb5d1-2991-4721-856d-4dae1e573184" (UID: "c51fb5d1-2991-4721-856d-4dae1e573184"). InnerVolumeSpecName "kube-api-access-rzrbm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:28:39.216171 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:39.216136 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzrbm\" (UniqueName: \"kubernetes.io/projected/c51fb5d1-2991-4721-856d-4dae1e573184-kube-api-access-rzrbm\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:28:39.813006 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:39.812968 2582 generic.go:356] "Generic (PLEG): container finished" podID="c51fb5d1-2991-4721-856d-4dae1e573184" containerID="c38b96669d5aa5773c014133eec144334ca5313082ed3b48f182ce1af3d575ed" exitCode=0 Jul 09 14:28:39.813421 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:39.813016 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nkdjd" Jul 09 14:28:39.813421 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:39.813055 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nkdjd" event={"ID":"c51fb5d1-2991-4721-856d-4dae1e573184","Type":"ContainerDied","Data":"c38b96669d5aa5773c014133eec144334ca5313082ed3b48f182ce1af3d575ed"} Jul 09 14:28:39.813421 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:39.813108 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nkdjd" event={"ID":"c51fb5d1-2991-4721-856d-4dae1e573184","Type":"ContainerDied","Data":"71843fb500c031e2f6bf81d97f2d0727228f472e04cf5691143bab0757fddb94"} Jul 09 14:28:39.813421 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:39.813131 2582 scope.go:117] "RemoveContainer" containerID="c38b96669d5aa5773c014133eec144334ca5313082ed3b48f182ce1af3d575ed" Jul 09 14:28:39.826618 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:39.826597 2582 scope.go:117] "RemoveContainer" containerID="c38b96669d5aa5773c014133eec144334ca5313082ed3b48f182ce1af3d575ed" Jul 09 14:28:39.826945 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:28:39.826925 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c38b96669d5aa5773c014133eec144334ca5313082ed3b48f182ce1af3d575ed\": container with ID starting with c38b96669d5aa5773c014133eec144334ca5313082ed3b48f182ce1af3d575ed not found: ID does not exist" containerID="c38b96669d5aa5773c014133eec144334ca5313082ed3b48f182ce1af3d575ed" Jul 09 14:28:39.827000 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:39.826954 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c38b96669d5aa5773c014133eec144334ca5313082ed3b48f182ce1af3d575ed"} err="failed to get container status \"c38b96669d5aa5773c014133eec144334ca5313082ed3b48f182ce1af3d575ed\": rpc error: code = NotFound desc = could not find container \"c38b96669d5aa5773c014133eec144334ca5313082ed3b48f182ce1af3d575ed\": container with ID starting with c38b96669d5aa5773c014133eec144334ca5313082ed3b48f182ce1af3d575ed not found: ID does not exist" Jul 09 14:28:39.836966 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:39.836941 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nkdjd"] Jul 09 14:28:39.843077 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:39.843053 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nkdjd"] Jul 09 14:28:40.036767 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:40.036736 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c51fb5d1-2991-4721-856d-4dae1e573184" path="/var/lib/kubelet/pods/c51fb5d1-2991-4721-856d-4dae1e573184/volumes" Jul 09 14:28:46.429428 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:46.429391 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-pjhd7" Jul 09 14:28:46.429844 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:46.429448 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-pjhd7" Jul 09 14:28:46.452051 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:46.452025 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-pjhd7" Jul 09 14:28:46.862568 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:28:46.862539 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-pjhd7" Jul 09 14:29:07.247321 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:07.247287 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-skzpv"] Jul 09 14:29:07.247750 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:07.247559 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c51fb5d1-2991-4721-856d-4dae1e573184" containerName="registry-server" Jul 09 14:29:07.247750 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:07.247569 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c51fb5d1-2991-4721-856d-4dae1e573184" containerName="registry-server" Jul 09 14:29:07.247750 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:07.247624 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c51fb5d1-2991-4721-856d-4dae1e573184" containerName="registry-server" Jul 09 14:29:07.255562 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:07.255539 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-skzpv" Jul 09 14:29:07.259076 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:07.259053 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-lzkv6\"" Jul 09 14:29:07.259201 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:07.259085 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 14:29:07.262539 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:07.262512 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-skzpv"] Jul 09 14:29:07.333310 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:07.333282 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrfw9\" (UniqueName: \"kubernetes.io/projected/2e596c25-02be-45a3-a7d2-aa76614bee03-kube-api-access-xrfw9\") pod \"dns-operator-controller-manager-864dd78c4c-skzpv\" (UID: \"2e596c25-02be-45a3-a7d2-aa76614bee03\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-skzpv" Jul 09 14:29:07.434399 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:07.434370 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrfw9\" (UniqueName: \"kubernetes.io/projected/2e596c25-02be-45a3-a7d2-aa76614bee03-kube-api-access-xrfw9\") pod \"dns-operator-controller-manager-864dd78c4c-skzpv\" (UID: \"2e596c25-02be-45a3-a7d2-aa76614bee03\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-skzpv" Jul 09 14:29:07.450914 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:07.450888 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrfw9\" (UniqueName: \"kubernetes.io/projected/2e596c25-02be-45a3-a7d2-aa76614bee03-kube-api-access-xrfw9\") pod \"dns-operator-controller-manager-864dd78c4c-skzpv\" (UID: \"2e596c25-02be-45a3-a7d2-aa76614bee03\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-skzpv" Jul 09 14:29:07.573524 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:07.573493 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-skzpv" Jul 09 14:29:07.709814 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:07.709758 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-skzpv"] Jul 09 14:29:07.712794 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:29:07.712767 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e596c25_02be_45a3_a7d2_aa76614bee03.slice/crio-0ece4b01b86cb5211370f3fd6ce87dc8fbc781764564370dddb350f040e1eb34 WatchSource:0}: Error finding container 0ece4b01b86cb5211370f3fd6ce87dc8fbc781764564370dddb350f040e1eb34: Status 404 returned error can't find the container with id 0ece4b01b86cb5211370f3fd6ce87dc8fbc781764564370dddb350f040e1eb34 Jul 09 14:29:07.924944 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:07.924832 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-skzpv" event={"ID":"2e596c25-02be-45a3-a7d2-aa76614bee03","Type":"ContainerStarted","Data":"0ece4b01b86cb5211370f3fd6ce87dc8fbc781764564370dddb350f040e1eb34"} Jul 09 14:29:10.939893 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:10.939846 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-skzpv" event={"ID":"2e596c25-02be-45a3-a7d2-aa76614bee03","Type":"ContainerStarted","Data":"da42b3ba87db0d3dc596625148b932195061ff06623801e0b69cc559b8877c0d"} Jul 09 14:29:10.940336 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:10.940013 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-skzpv" Jul 09 14:29:10.964680 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:10.964632 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-skzpv" podStartSLOduration=1.623313688 podStartE2EDuration="3.964617995s" totalImagesPullingTime="2.341304307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:29:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:29:10.960734844 +0000 UTC m=+501.326829231" watchObservedRunningTime="2026-07-09 14:29:10.964617995 +0000 UTC m=+501.330712381" Jul 09 14:29:12.655628 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:12.655581 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz"] Jul 09 14:29:12.662947 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:12.662916 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:12.665102 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:12.665075 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz"] Jul 09 14:29:12.666046 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:12.666027 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hgl5d\"" Jul 09 14:29:12.777417 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:12.777386 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/369d43e7-506f-4ca5-8599-8962b690a4c3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qzmz\" (UID: \"369d43e7-506f-4ca5-8599-8962b690a4c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:12.777542 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:12.777429 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tr52\" (UniqueName: \"kubernetes.io/projected/369d43e7-506f-4ca5-8599-8962b690a4c3-kube-api-access-8tr52\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qzmz\" (UID: \"369d43e7-506f-4ca5-8599-8962b690a4c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:12.878746 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:12.878704 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/369d43e7-506f-4ca5-8599-8962b690a4c3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qzmz\" (UID: \"369d43e7-506f-4ca5-8599-8962b690a4c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:12.878947 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:12.878765 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tr52\" (UniqueName: \"kubernetes.io/projected/369d43e7-506f-4ca5-8599-8962b690a4c3-kube-api-access-8tr52\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qzmz\" (UID: \"369d43e7-506f-4ca5-8599-8962b690a4c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:12.879130 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:12.879109 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/369d43e7-506f-4ca5-8599-8962b690a4c3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qzmz\" (UID: \"369d43e7-506f-4ca5-8599-8962b690a4c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:12.902763 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:12.902733 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tr52\" (UniqueName: \"kubernetes.io/projected/369d43e7-506f-4ca5-8599-8962b690a4c3-kube-api-access-8tr52\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5qzmz\" (UID: \"369d43e7-506f-4ca5-8599-8962b690a4c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:12.975594 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:12.975510 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:13.139701 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:13.139669 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz"] Jul 09 14:29:13.143622 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:29:13.143596 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod369d43e7_506f_4ca5_8599_8962b690a4c3.slice/crio-423e84ca4434e6a509d3364ac405b5c1799d72754e3d4ae5caaaec5db5d24452 WatchSource:0}: Error finding container 423e84ca4434e6a509d3364ac405b5c1799d72754e3d4ae5caaaec5db5d24452: Status 404 returned error can't find the container with id 423e84ca4434e6a509d3364ac405b5c1799d72754e3d4ae5caaaec5db5d24452 Jul 09 14:29:13.953467 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:13.953434 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" event={"ID":"369d43e7-506f-4ca5-8599-8962b690a4c3","Type":"ContainerStarted","Data":"423e84ca4434e6a509d3364ac405b5c1799d72754e3d4ae5caaaec5db5d24452"} Jul 09 14:29:16.314908 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:16.314854 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-q5t8t"] Jul 09 14:29:16.318305 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:16.318281 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q5t8t" Jul 09 14:29:16.322474 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:16.322458 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-v2ls9\"" Jul 09 14:29:16.345414 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:16.345391 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-q5t8t"] Jul 09 14:29:16.410810 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:16.410774 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rwkn\" (UniqueName: \"kubernetes.io/projected/1201ce06-31e0-46ac-8a6a-f816710be1dc-kube-api-access-2rwkn\") pod \"authorino-operator-75cc8d5cb7-q5t8t\" (UID: \"1201ce06-31e0-46ac-8a6a-f816710be1dc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-q5t8t" Jul 09 14:29:16.511481 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:16.511446 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rwkn\" (UniqueName: \"kubernetes.io/projected/1201ce06-31e0-46ac-8a6a-f816710be1dc-kube-api-access-2rwkn\") pod \"authorino-operator-75cc8d5cb7-q5t8t\" (UID: \"1201ce06-31e0-46ac-8a6a-f816710be1dc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-q5t8t" Jul 09 14:29:16.527112 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:16.527086 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rwkn\" (UniqueName: \"kubernetes.io/projected/1201ce06-31e0-46ac-8a6a-f816710be1dc-kube-api-access-2rwkn\") pod \"authorino-operator-75cc8d5cb7-q5t8t\" (UID: \"1201ce06-31e0-46ac-8a6a-f816710be1dc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-q5t8t" Jul 09 14:29:16.628415 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:16.628339 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q5t8t" Jul 09 14:29:16.777305 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:16.777270 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-q5t8t"] Jul 09 14:29:16.779925 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:29:16.779894 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1201ce06_31e0_46ac_8a6a_f816710be1dc.slice/crio-72f930d60c843297f5d1624a81ab482458e375547b91f68a962e864f97c3b706 WatchSource:0}: Error finding container 72f930d60c843297f5d1624a81ab482458e375547b91f68a962e864f97c3b706: Status 404 returned error can't find the container with id 72f930d60c843297f5d1624a81ab482458e375547b91f68a962e864f97c3b706 Jul 09 14:29:16.969414 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:16.969330 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q5t8t" event={"ID":"1201ce06-31e0-46ac-8a6a-f816710be1dc","Type":"ContainerStarted","Data":"72f930d60c843297f5d1624a81ab482458e375547b91f68a962e864f97c3b706"} Jul 09 14:29:19.039734 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:19.039695 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq"] Jul 09 14:29:19.043685 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:19.043660 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:19.050224 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:19.050198 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-7q9fc\"" Jul 09 14:29:19.061340 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:19.061316 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq"] Jul 09 14:29:19.135354 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:19.135319 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8gbp\" (UniqueName: \"kubernetes.io/projected/5001c8f1-8208-449d-87bf-959aca6d0fde-kube-api-access-v8gbp\") pod \"limitador-operator-controller-manager-859d7b78bd-cpvbq\" (UID: \"5001c8f1-8208-449d-87bf-959aca6d0fde\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:19.236291 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:19.236243 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8gbp\" (UniqueName: \"kubernetes.io/projected/5001c8f1-8208-449d-87bf-959aca6d0fde-kube-api-access-v8gbp\") pod \"limitador-operator-controller-manager-859d7b78bd-cpvbq\" (UID: \"5001c8f1-8208-449d-87bf-959aca6d0fde\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:19.251007 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:19.250978 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8gbp\" (UniqueName: \"kubernetes.io/projected/5001c8f1-8208-449d-87bf-959aca6d0fde-kube-api-access-v8gbp\") pod \"limitador-operator-controller-manager-859d7b78bd-cpvbq\" (UID: \"5001c8f1-8208-449d-87bf-959aca6d0fde\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:19.357247 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:19.357211 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:20.407716 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:20.407575 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq"] Jul 09 14:29:20.410435 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:29:20.410408 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5001c8f1_8208_449d_87bf_959aca6d0fde.slice/crio-89eeedbfaea68fb484abbb73d916899ddfdeb134a464a9561e28ed944c989c9a WatchSource:0}: Error finding container 89eeedbfaea68fb484abbb73d916899ddfdeb134a464a9561e28ed944c989c9a: Status 404 returned error can't find the container with id 89eeedbfaea68fb484abbb73d916899ddfdeb134a464a9561e28ed944c989c9a Jul 09 14:29:20.990213 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:20.990175 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q5t8t" event={"ID":"1201ce06-31e0-46ac-8a6a-f816710be1dc","Type":"ContainerStarted","Data":"1c20dbe12b07ce2b0c669465369383fe8fb4a0e35b880204686d03e8bb4bf162"} Jul 09 14:29:20.991926 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:20.991886 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" event={"ID":"5001c8f1-8208-449d-87bf-959aca6d0fde","Type":"ContainerStarted","Data":"89eeedbfaea68fb484abbb73d916899ddfdeb134a464a9561e28ed944c989c9a"} Jul 09 14:29:20.993901 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:20.993843 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" event={"ID":"369d43e7-506f-4ca5-8599-8962b690a4c3","Type":"ContainerStarted","Data":"4ccdc2f9213aecfdd9f2744fe6ac2f18d034b5d1ff525804f43c5530edc33ef7"} Jul 09 14:29:20.994067 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:20.994047 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:21.012435 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:21.012383 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q5t8t" podStartSLOduration=1.5241512849999999 podStartE2EDuration="5.012368084s" totalImagesPullingTime="3.488216799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:29:21.011693999 +0000 UTC m=+511.377788386" watchObservedRunningTime="2026-07-09 14:29:21.012368084 +0000 UTC m=+511.378462828" Jul 09 14:29:21.039044 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:21.038987 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" podStartSLOduration=1.9138880569999999 podStartE2EDuration="9.038965861s" totalImagesPullingTime="7.125077804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:29:21.036129111 +0000 UTC m=+511.402223498" watchObservedRunningTime="2026-07-09 14:29:21.038965861 +0000 UTC m=+511.405060249" Jul 09 14:29:21.946667 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:21.946633 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-skzpv" Jul 09 14:29:22.002047 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:22.002022 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q5t8t" Jul 09 14:29:23.008097 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.008055 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" event={"ID":"5001c8f1-8208-449d-87bf-959aca6d0fde","Type":"ContainerStarted","Data":"d7107ed77e2d2a08e998d4815384a484fe61cbea57ca7ab7f38c7e896e31b302"} Jul 09 14:29:23.008523 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.008157 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:23.044045 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.043999 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" podStartSLOduration=2.454444996 podStartE2EDuration="4.043984434s" totalImagesPullingTime="1.589539438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:29:23.042849801 +0000 UTC m=+513.408944187" watchObservedRunningTime="2026-07-09 14:29:23.043984434 +0000 UTC m=+513.410078863" Jul 09 14:29:23.414746 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.414714 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k"] Jul 09 14:29:23.418142 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.418115 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" Jul 09 14:29:23.420618 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.420598 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 09 14:29:23.420618 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.420612 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-g7ht8\"" Jul 09 14:29:23.420919 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.420903 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 09 14:29:23.430594 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.430571 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k"] Jul 09 14:29:23.572480 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.572443 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/359be0dc-ada2-4058-aa4d-bee8b34419d3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-djh9k\" (UID: \"359be0dc-ada2-4058-aa4d-bee8b34419d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" Jul 09 14:29:23.572480 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.572484 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bthbn\" (UniqueName: \"kubernetes.io/projected/359be0dc-ada2-4058-aa4d-bee8b34419d3-kube-api-access-bthbn\") pod \"kuadrant-console-plugin-66b5774985-djh9k\" (UID: \"359be0dc-ada2-4058-aa4d-bee8b34419d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" Jul 09 14:29:23.572688 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.572521 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/359be0dc-ada2-4058-aa4d-bee8b34419d3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-djh9k\" (UID: \"359be0dc-ada2-4058-aa4d-bee8b34419d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" Jul 09 14:29:23.673064 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.672990 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/359be0dc-ada2-4058-aa4d-bee8b34419d3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-djh9k\" (UID: \"359be0dc-ada2-4058-aa4d-bee8b34419d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" Jul 09 14:29:23.673218 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.673068 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/359be0dc-ada2-4058-aa4d-bee8b34419d3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-djh9k\" (UID: \"359be0dc-ada2-4058-aa4d-bee8b34419d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" Jul 09 14:29:23.673218 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.673089 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bthbn\" (UniqueName: \"kubernetes.io/projected/359be0dc-ada2-4058-aa4d-bee8b34419d3-kube-api-access-bthbn\") pod \"kuadrant-console-plugin-66b5774985-djh9k\" (UID: \"359be0dc-ada2-4058-aa4d-bee8b34419d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" Jul 09 14:29:23.673320 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:23.673227 2582 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 09 14:29:23.673320 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:23.673301 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/359be0dc-ada2-4058-aa4d-bee8b34419d3-plugin-serving-cert podName:359be0dc-ada2-4058-aa4d-bee8b34419d3 nodeName:}" failed. No retries permitted until 2026-07-09 14:29:24.173280253 +0000 UTC m=+514.539374623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/359be0dc-ada2-4058-aa4d-bee8b34419d3-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-djh9k" (UID: "359be0dc-ada2-4058-aa4d-bee8b34419d3") : secret "plugin-serving-cert" not found Jul 09 14:29:23.673623 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.673603 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/359be0dc-ada2-4058-aa4d-bee8b34419d3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-djh9k\" (UID: \"359be0dc-ada2-4058-aa4d-bee8b34419d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" Jul 09 14:29:23.682495 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:23.682469 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bthbn\" (UniqueName: \"kubernetes.io/projected/359be0dc-ada2-4058-aa4d-bee8b34419d3-kube-api-access-bthbn\") pod \"kuadrant-console-plugin-66b5774985-djh9k\" (UID: \"359be0dc-ada2-4058-aa4d-bee8b34419d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" Jul 09 14:29:24.177672 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:24.177636 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/359be0dc-ada2-4058-aa4d-bee8b34419d3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-djh9k\" (UID: \"359be0dc-ada2-4058-aa4d-bee8b34419d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" Jul 09 14:29:24.180321 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:24.180297 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/359be0dc-ada2-4058-aa4d-bee8b34419d3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-djh9k\" (UID: \"359be0dc-ada2-4058-aa4d-bee8b34419d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" Jul 09 14:29:24.327975 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:24.327931 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" Jul 09 14:29:24.459244 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:24.459216 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k"] Jul 09 14:29:24.461098 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:29:24.461069 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod359be0dc_ada2_4058_aa4d_bee8b34419d3.slice/crio-bf659697f0449724abd48b95c3b5b76a6482056ae94ca4eb318e1c49a3cf573d WatchSource:0}: Error finding container bf659697f0449724abd48b95c3b5b76a6482056ae94ca4eb318e1c49a3cf573d: Status 404 returned error can't find the container with id bf659697f0449724abd48b95c3b5b76a6482056ae94ca4eb318e1c49a3cf573d Jul 09 14:29:25.016917 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:25.016861 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" event={"ID":"359be0dc-ada2-4058-aa4d-bee8b34419d3","Type":"ContainerStarted","Data":"bf659697f0449724abd48b95c3b5b76a6482056ae94ca4eb318e1c49a3cf573d"} Jul 09 14:29:32.005002 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:32.004964 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:33.010819 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.010789 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-q5t8t" Jul 09 14:29:33.885036 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.885003 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz"] Jul 09 14:29:33.885231 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.885196 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" podUID="369d43e7-506f-4ca5-8599-8962b690a4c3" containerName="manager" containerID="cri-o://4ccdc2f9213aecfdd9f2744fe6ac2f18d034b5d1ff525804f43c5530edc33ef7" gracePeriod=2 Jul 09 14:29:33.894778 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.894720 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz"] Jul 09 14:29:33.910805 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.910780 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq"] Jul 09 14:29:33.911278 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.911019 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" podUID="5001c8f1-8208-449d-87bf-959aca6d0fde" containerName="manager" containerID="cri-o://d7107ed77e2d2a08e998d4815384a484fe61cbea57ca7ab7f38c7e896e31b302" gracePeriod=2 Jul 09 14:29:33.913768 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:33.913680 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5qzmz\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="369d43e7-506f-4ca5-8599-8962b690a4c3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:33.913768 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.913719 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:33.944962 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:33.944131 2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5001c8f1-8208-449d-87bf-959aca6d0fde\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-cpvbq\": pods \"limitador-operator-controller-manager-859d7b78bd-cpvbq\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:33.947217 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.947188 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq"] Jul 09 14:29:33.951460 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:33.951427 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5qzmz\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="369d43e7-506f-4ca5-8599-8962b690a4c3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:33.953773 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:33.953740 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cpvbq\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="5001c8f1-8208-449d-87bf-959aca6d0fde" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:33.984947 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.984479 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-czvbd"] Jul 09 14:29:33.986513 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.986355 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="369d43e7-506f-4ca5-8599-8962b690a4c3" containerName="manager" Jul 09 14:29:33.986513 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.986379 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="369d43e7-506f-4ca5-8599-8962b690a4c3" containerName="manager" Jul 09 14:29:33.986513 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.986398 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5001c8f1-8208-449d-87bf-959aca6d0fde" containerName="manager" Jul 09 14:29:33.986513 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.986407 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5001c8f1-8208-449d-87bf-959aca6d0fde" containerName="manager" Jul 09 14:29:33.986513 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.986493 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5001c8f1-8208-449d-87bf-959aca6d0fde" containerName="manager" Jul 09 14:29:33.986513 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.986506 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="369d43e7-506f-4ca5-8599-8962b690a4c3" containerName="manager" Jul 09 14:29:33.992978 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.992954 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7"] Jul 09 14:29:33.993116 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:33.993097 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-czvbd" Jul 09 14:29:34.000676 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:34.000653 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" Jul 09 14:29:34.002067 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:34.002032 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cpvbq\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="5001c8f1-8208-449d-87bf-959aca6d0fde" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:34.024422 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:34.024398 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-czvbd"] Jul 09 14:29:34.046450 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:34.046422 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7"] Jul 09 14:29:34.065483 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:34.065443 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5qzmz\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="369d43e7-506f-4ca5-8599-8962b690a4c3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:34.167752 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:34.167661 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lvt5\" (UniqueName: \"kubernetes.io/projected/50dce30c-480e-4604-8de3-450f22b9a462-kube-api-access-2lvt5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kmtr7\" (UID: \"50dce30c-480e-4604-8de3-450f22b9a462\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" Jul 09 14:29:34.167932 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:34.167810 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9ltq\" (UniqueName: \"kubernetes.io/projected/7cccb007-377a-4a3c-8339-ac57ab28c5e1-kube-api-access-q9ltq\") pod \"limitador-operator-controller-manager-859d7b78bd-czvbd\" (UID: \"7cccb007-377a-4a3c-8339-ac57ab28c5e1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-czvbd" Jul 09 14:29:34.167932 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:34.167856 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50dce30c-480e-4604-8de3-450f22b9a462-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kmtr7\" (UID: \"50dce30c-480e-4604-8de3-450f22b9a462\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" Jul 09 14:29:34.269021 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:34.268980 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9ltq\" (UniqueName: \"kubernetes.io/projected/7cccb007-377a-4a3c-8339-ac57ab28c5e1-kube-api-access-q9ltq\") pod \"limitador-operator-controller-manager-859d7b78bd-czvbd\" (UID: \"7cccb007-377a-4a3c-8339-ac57ab28c5e1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-czvbd" Jul 09 14:29:34.269021 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:34.269024 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50dce30c-480e-4604-8de3-450f22b9a462-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kmtr7\" (UID: \"50dce30c-480e-4604-8de3-450f22b9a462\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" Jul 09 14:29:34.269275 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:34.269046 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lvt5\" (UniqueName: \"kubernetes.io/projected/50dce30c-480e-4604-8de3-450f22b9a462-kube-api-access-2lvt5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kmtr7\" (UID: \"50dce30c-480e-4604-8de3-450f22b9a462\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" Jul 09 14:29:34.269426 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:34.269402 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50dce30c-480e-4604-8de3-450f22b9a462-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kmtr7\" (UID: \"50dce30c-480e-4604-8de3-450f22b9a462\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" Jul 09 14:29:34.278749 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:34.278714 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lvt5\" (UniqueName: \"kubernetes.io/projected/50dce30c-480e-4604-8de3-450f22b9a462-kube-api-access-2lvt5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kmtr7\" (UID: \"50dce30c-480e-4604-8de3-450f22b9a462\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" Jul 09 14:29:34.278914 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:34.278762 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9ltq\" (UniqueName: \"kubernetes.io/projected/7cccb007-377a-4a3c-8339-ac57ab28c5e1-kube-api-access-q9ltq\") pod \"limitador-operator-controller-manager-859d7b78bd-czvbd\" (UID: \"7cccb007-377a-4a3c-8339-ac57ab28c5e1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-czvbd" Jul 09 14:29:34.304649 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:34.304616 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-czvbd" Jul 09 14:29:34.314778 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:34.314749 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" Jul 09 14:29:35.157214 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:35.157173 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z"] Jul 09 14:29:35.169673 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:35.169649 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" Jul 09 14:29:35.177236 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:35.177206 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z"] Jul 09 14:29:35.276824 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:35.276780 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a14878ce-f095-4066-993c-daad867c5d44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h8s2z\" (UID: \"a14878ce-f095-4066-993c-daad867c5d44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" Jul 09 14:29:35.277037 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:35.276955 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppzv5\" (UniqueName: \"kubernetes.io/projected/a14878ce-f095-4066-993c-daad867c5d44-kube-api-access-ppzv5\") pod \"kuadrant-operator-controller-manager-86878c769f-h8s2z\" (UID: \"a14878ce-f095-4066-993c-daad867c5d44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" Jul 09 14:29:35.377689 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:35.377655 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppzv5\" (UniqueName: \"kubernetes.io/projected/a14878ce-f095-4066-993c-daad867c5d44-kube-api-access-ppzv5\") pod \"kuadrant-operator-controller-manager-86878c769f-h8s2z\" (UID: \"a14878ce-f095-4066-993c-daad867c5d44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" Jul 09 14:29:35.377854 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:35.377751 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a14878ce-f095-4066-993c-daad867c5d44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h8s2z\" (UID: \"a14878ce-f095-4066-993c-daad867c5d44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" Jul 09 14:29:35.378152 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:35.378126 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a14878ce-f095-4066-993c-daad867c5d44-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h8s2z\" (UID: \"a14878ce-f095-4066-993c-daad867c5d44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" Jul 09 14:29:35.397620 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:35.397591 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppzv5\" (UniqueName: \"kubernetes.io/projected/a14878ce-f095-4066-993c-daad867c5d44-kube-api-access-ppzv5\") pod \"kuadrant-operator-controller-manager-86878c769f-h8s2z\" (UID: \"a14878ce-f095-4066-993c-daad867c5d44\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" Jul 09 14:29:35.483721 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:35.483645 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" Jul 09 14:29:46.743975 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:46.743895 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:46.749006 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:46.748968 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cpvbq\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="5001c8f1-8208-449d-87bf-959aca6d0fde" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:46.753883 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:46.753845 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:46.757535 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:46.757505 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5qzmz\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="369d43e7-506f-4ca5-8599-8962b690a4c3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:46.759602 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:46.759568 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cpvbq\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="5001c8f1-8208-449d-87bf-959aca6d0fde" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:46.764223 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:46.764202 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7"] Jul 09 14:29:46.766932 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:29:46.766908 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50dce30c_480e_4604_8de3_450f22b9a462.slice/crio-72e73b1a66d0bfc7230e736fc3307fc7a4d97272156e57ee0af0ffd2ff151305 WatchSource:0}: Error finding container 72e73b1a66d0bfc7230e736fc3307fc7a4d97272156e57ee0af0ffd2ff151305: Status 404 returned error can't find the container with id 72e73b1a66d0bfc7230e736fc3307fc7a4d97272156e57ee0af0ffd2ff151305 Jul 09 14:29:46.779265 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:46.779247 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8gbp\" (UniqueName: \"kubernetes.io/projected/5001c8f1-8208-449d-87bf-959aca6d0fde-kube-api-access-v8gbp\") pod \"5001c8f1-8208-449d-87bf-959aca6d0fde\" (UID: \"5001c8f1-8208-449d-87bf-959aca6d0fde\") " Jul 09 14:29:46.779371 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:46.779298 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tr52\" (UniqueName: \"kubernetes.io/projected/369d43e7-506f-4ca5-8599-8962b690a4c3-kube-api-access-8tr52\") pod \"369d43e7-506f-4ca5-8599-8962b690a4c3\" (UID: \"369d43e7-506f-4ca5-8599-8962b690a4c3\") " Jul 09 14:29:46.779371 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:46.779335 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/369d43e7-506f-4ca5-8599-8962b690a4c3-extensions-socket-volume\") pod \"369d43e7-506f-4ca5-8599-8962b690a4c3\" (UID: \"369d43e7-506f-4ca5-8599-8962b690a4c3\") " Jul 09 14:29:46.779889 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:46.779848 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/369d43e7-506f-4ca5-8599-8962b690a4c3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "369d43e7-506f-4ca5-8599-8962b690a4c3" (UID: "369d43e7-506f-4ca5-8599-8962b690a4c3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:29:46.781327 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:46.781300 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5001c8f1-8208-449d-87bf-959aca6d0fde-kube-api-access-v8gbp" (OuterVolumeSpecName: "kube-api-access-v8gbp") pod "5001c8f1-8208-449d-87bf-959aca6d0fde" (UID: "5001c8f1-8208-449d-87bf-959aca6d0fde"). InnerVolumeSpecName "kube-api-access-v8gbp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:29:46.781410 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:46.781375 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/369d43e7-506f-4ca5-8599-8962b690a4c3-kube-api-access-8tr52" (OuterVolumeSpecName: "kube-api-access-8tr52") pod "369d43e7-506f-4ca5-8599-8962b690a4c3" (UID: "369d43e7-506f-4ca5-8599-8962b690a4c3"). InnerVolumeSpecName "kube-api-access-8tr52". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:29:46.880360 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:46.880328 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tr52\" (UniqueName: \"kubernetes.io/projected/369d43e7-506f-4ca5-8599-8962b690a4c3-kube-api-access-8tr52\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:29:46.880360 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:46.880357 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/369d43e7-506f-4ca5-8599-8962b690a4c3-extensions-socket-volume\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:29:46.880555 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:46.880369 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8gbp\" (UniqueName: \"kubernetes.io/projected/5001c8f1-8208-449d-87bf-959aca6d0fde-kube-api-access-v8gbp\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:29:46.977018 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:46.976992 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-czvbd"] Jul 09 14:29:46.979849 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:29:46.979820 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cccb007_377a_4a3c_8339_ac57ab28c5e1.slice/crio-2012c031ddc9dad454afd0b8a7f5a18e646bdbc14e151f88dad896d38123380c WatchSource:0}: Error finding container 2012c031ddc9dad454afd0b8a7f5a18e646bdbc14e151f88dad896d38123380c: Status 404 returned error can't find the container with id 2012c031ddc9dad454afd0b8a7f5a18e646bdbc14e151f88dad896d38123380c Jul 09 14:29:46.980335 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:46.980310 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z"] Jul 09 14:29:46.983517 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:29:46.983496 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda14878ce_f095_4066_993c_daad867c5d44.slice/crio-471162de8ac7a85968ecb29d423c8615e8f89c3b24f61aa34305fdf4be6060ba WatchSource:0}: Error finding container 471162de8ac7a85968ecb29d423c8615e8f89c3b24f61aa34305fdf4be6060ba: Status 404 returned error can't find the container with id 471162de8ac7a85968ecb29d423c8615e8f89c3b24f61aa34305fdf4be6060ba Jul 09 14:29:47.127772 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.127731 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" event={"ID":"50dce30c-480e-4604-8de3-450f22b9a462","Type":"ContainerStarted","Data":"1bbd19778f5b97f0c81cc03fc2b7c1c491cec08ee48db103af6e55714a3963f8"} Jul 09 14:29:47.128016 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.127828 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" event={"ID":"50dce30c-480e-4604-8de3-450f22b9a462","Type":"ContainerStarted","Data":"72e73b1a66d0bfc7230e736fc3307fc7a4d97272156e57ee0af0ffd2ff151305"} Jul 09 14:29:47.128016 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.127857 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" Jul 09 14:29:47.129403 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.129355 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" event={"ID":"a14878ce-f095-4066-993c-daad867c5d44","Type":"ContainerStarted","Data":"8a3978466292a2ca01c9aef25a4c981edc404a55c6c3abd1a217c8a7403b3ea1"} Jul 09 14:29:47.129518 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.129421 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" event={"ID":"a14878ce-f095-4066-993c-daad867c5d44","Type":"ContainerStarted","Data":"471162de8ac7a85968ecb29d423c8615e8f89c3b24f61aa34305fdf4be6060ba"} Jul 09 14:29:47.129518 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.129464 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" Jul 09 14:29:47.130662 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.130639 2582 generic.go:356] "Generic (PLEG): container finished" podID="5001c8f1-8208-449d-87bf-959aca6d0fde" containerID="d7107ed77e2d2a08e998d4815384a484fe61cbea57ca7ab7f38c7e896e31b302" exitCode=0 Jul 09 14:29:47.130757 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.130685 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:47.130757 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.130735 2582 scope.go:117] "RemoveContainer" containerID="d7107ed77e2d2a08e998d4815384a484fe61cbea57ca7ab7f38c7e896e31b302" Jul 09 14:29:47.132421 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.132400 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-czvbd" event={"ID":"7cccb007-377a-4a3c-8339-ac57ab28c5e1","Type":"ContainerStarted","Data":"221fb3e69bbc819ac0eb7f6f1f5aaf936aa5e5c6d27fa063532e33e0fedbc433"} Jul 09 14:29:47.132535 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.132435 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-czvbd" event={"ID":"7cccb007-377a-4a3c-8339-ac57ab28c5e1","Type":"ContainerStarted","Data":"2012c031ddc9dad454afd0b8a7f5a18e646bdbc14e151f88dad896d38123380c"} Jul 09 14:29:47.132535 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.132500 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-czvbd" Jul 09 14:29:47.133784 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.133761 2582 generic.go:356] "Generic (PLEG): container finished" podID="369d43e7-506f-4ca5-8599-8962b690a4c3" containerID="4ccdc2f9213aecfdd9f2744fe6ac2f18d034b5d1ff525804f43c5530edc33ef7" exitCode=0 Jul 09 14:29:47.133917 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.133817 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:47.135557 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.135533 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" event={"ID":"359be0dc-ada2-4058-aa4d-bee8b34419d3","Type":"ContainerStarted","Data":"88f33b649c1ed632b069d0c2e2ff8f95a7551beada3967b022af41ddc82198c8"} Jul 09 14:29:47.145399 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.145378 2582 scope.go:117] "RemoveContainer" containerID="d7107ed77e2d2a08e998d4815384a484fe61cbea57ca7ab7f38c7e896e31b302" Jul 09 14:29:47.145704 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:47.145677 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7107ed77e2d2a08e998d4815384a484fe61cbea57ca7ab7f38c7e896e31b302\": container with ID starting with d7107ed77e2d2a08e998d4815384a484fe61cbea57ca7ab7f38c7e896e31b302 not found: ID does not exist" containerID="d7107ed77e2d2a08e998d4815384a484fe61cbea57ca7ab7f38c7e896e31b302" Jul 09 14:29:47.145773 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.145718 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7107ed77e2d2a08e998d4815384a484fe61cbea57ca7ab7f38c7e896e31b302"} err="failed to get container status \"d7107ed77e2d2a08e998d4815384a484fe61cbea57ca7ab7f38c7e896e31b302\": rpc error: code = NotFound desc = could not find container \"d7107ed77e2d2a08e998d4815384a484fe61cbea57ca7ab7f38c7e896e31b302\": container with ID starting with d7107ed77e2d2a08e998d4815384a484fe61cbea57ca7ab7f38c7e896e31b302 not found: ID does not exist" Jul 09 14:29:47.145773 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.145746 2582 scope.go:117] "RemoveContainer" containerID="4ccdc2f9213aecfdd9f2744fe6ac2f18d034b5d1ff525804f43c5530edc33ef7" Jul 09 14:29:47.151936 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.151894 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" podStartSLOduration=14.151858349 podStartE2EDuration="14.151858349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:29:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:29:47.150299567 +0000 UTC m=+537.516393991" watchObservedRunningTime="2026-07-09 14:29:47.151858349 +0000 UTC m=+537.517952734" Jul 09 14:29:47.154629 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:47.154603 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cpvbq\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="5001c8f1-8208-449d-87bf-959aca6d0fde" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:47.155129 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.155114 2582 scope.go:117] "RemoveContainer" containerID="4ccdc2f9213aecfdd9f2744fe6ac2f18d034b5d1ff525804f43c5530edc33ef7" Jul 09 14:29:47.155386 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:47.155369 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ccdc2f9213aecfdd9f2744fe6ac2f18d034b5d1ff525804f43c5530edc33ef7\": container with ID starting with 4ccdc2f9213aecfdd9f2744fe6ac2f18d034b5d1ff525804f43c5530edc33ef7 not found: ID does not exist" containerID="4ccdc2f9213aecfdd9f2744fe6ac2f18d034b5d1ff525804f43c5530edc33ef7" Jul 09 14:29:47.155439 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.155391 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ccdc2f9213aecfdd9f2744fe6ac2f18d034b5d1ff525804f43c5530edc33ef7"} err="failed to get container status \"4ccdc2f9213aecfdd9f2744fe6ac2f18d034b5d1ff525804f43c5530edc33ef7\": rpc error: code = NotFound desc = could not find container \"4ccdc2f9213aecfdd9f2744fe6ac2f18d034b5d1ff525804f43c5530edc33ef7\": container with ID starting with 4ccdc2f9213aecfdd9f2744fe6ac2f18d034b5d1ff525804f43c5530edc33ef7 not found: ID does not exist" Jul 09 14:29:47.156566 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:47.156539 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5qzmz\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="369d43e7-506f-4ca5-8599-8962b690a4c3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:47.158574 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:47.158552 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5qzmz\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="369d43e7-506f-4ca5-8599-8962b690a4c3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5qzmz" Jul 09 14:29:47.189636 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:47.189605 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cpvbq\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="5001c8f1-8208-449d-87bf-959aca6d0fde" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cpvbq" Jul 09 14:29:47.220299 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.220233 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-czvbd" podStartSLOduration=14.220217238 podStartE2EDuration="14.220217238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:29:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:29:47.217737918 +0000 UTC m=+537.583832305" watchObservedRunningTime="2026-07-09 14:29:47.220217238 +0000 UTC m=+537.586311624" Jul 09 14:29:47.221622 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.221450 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-djh9k" podStartSLOduration=2.040283088 podStartE2EDuration="24.221436248s" totalImagesPullingTime="22.18115316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:29:47.187551337 +0000 UTC m=+537.553645733" watchObservedRunningTime="2026-07-09 14:29:47.221436248 +0000 UTC m=+537.587530639" Jul 09 14:29:47.251532 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:47.251461 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" podStartSLOduration=12.251437801 podStartE2EDuration="12.251437801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:29:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:29:47.25080345 +0000 UTC m=+537.616897866" watchObservedRunningTime="2026-07-09 14:29:47.251437801 +0000 UTC m=+537.617532188" Jul 09 14:29:48.038379 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:48.038339 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="369d43e7-506f-4ca5-8599-8962b690a4c3" path="/var/lib/kubelet/pods/369d43e7-506f-4ca5-8599-8962b690a4c3/volumes" Jul 09 14:29:48.038851 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:48.038805 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5001c8f1-8208-449d-87bf-959aca6d0fde" path="/var/lib/kubelet/pods/5001c8f1-8208-449d-87bf-959aca6d0fde/volumes" Jul 09 14:29:58.145533 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:58.145498 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" Jul 09 14:29:58.146030 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:58.145751 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" Jul 09 14:29:58.146030 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:58.145906 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-czvbd" Jul 09 14:29:58.245268 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:58.245232 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7"] Jul 09 14:29:58.245442 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:58.245418 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" podUID="50dce30c-480e-4604-8de3-450f22b9a462" containerName="manager" containerID="cri-o://1bbd19778f5b97f0c81cc03fc2b7c1c491cec08ee48db103af6e55714a3963f8" gracePeriod=10 Jul 09 14:29:58.493833 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:58.493804 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" Jul 09 14:29:58.584928 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:58.584895 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50dce30c-480e-4604-8de3-450f22b9a462-extensions-socket-volume\") pod \"50dce30c-480e-4604-8de3-450f22b9a462\" (UID: \"50dce30c-480e-4604-8de3-450f22b9a462\") " Jul 09 14:29:58.585061 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:58.584951 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lvt5\" (UniqueName: \"kubernetes.io/projected/50dce30c-480e-4604-8de3-450f22b9a462-kube-api-access-2lvt5\") pod \"50dce30c-480e-4604-8de3-450f22b9a462\" (UID: \"50dce30c-480e-4604-8de3-450f22b9a462\") " Jul 09 14:29:58.585335 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:58.585308 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50dce30c-480e-4604-8de3-450f22b9a462-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "50dce30c-480e-4604-8de3-450f22b9a462" (UID: "50dce30c-480e-4604-8de3-450f22b9a462"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:29:58.587122 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:58.587101 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50dce30c-480e-4604-8de3-450f22b9a462-kube-api-access-2lvt5" (OuterVolumeSpecName: "kube-api-access-2lvt5") pod "50dce30c-480e-4604-8de3-450f22b9a462" (UID: "50dce30c-480e-4604-8de3-450f22b9a462"). InnerVolumeSpecName "kube-api-access-2lvt5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:29:58.686506 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:58.686429 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50dce30c-480e-4604-8de3-450f22b9a462-extensions-socket-volume\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:29:58.686506 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:58.686455 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lvt5\" (UniqueName: \"kubernetes.io/projected/50dce30c-480e-4604-8de3-450f22b9a462-kube-api-access-2lvt5\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:29:59.191190 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:59.191156 2582 generic.go:356] "Generic (PLEG): container finished" podID="50dce30c-480e-4604-8de3-450f22b9a462" containerID="1bbd19778f5b97f0c81cc03fc2b7c1c491cec08ee48db103af6e55714a3963f8" exitCode=0 Jul 09 14:29:59.191629 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:59.191198 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" Jul 09 14:29:59.191629 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:59.191239 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" event={"ID":"50dce30c-480e-4604-8de3-450f22b9a462","Type":"ContainerDied","Data":"1bbd19778f5b97f0c81cc03fc2b7c1c491cec08ee48db103af6e55714a3963f8"} Jul 09 14:29:59.191629 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:59.191297 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7" event={"ID":"50dce30c-480e-4604-8de3-450f22b9a462","Type":"ContainerDied","Data":"72e73b1a66d0bfc7230e736fc3307fc7a4d97272156e57ee0af0ffd2ff151305"} Jul 09 14:29:59.191629 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:59.191330 2582 scope.go:117] "RemoveContainer" containerID="1bbd19778f5b97f0c81cc03fc2b7c1c491cec08ee48db103af6e55714a3963f8" Jul 09 14:29:59.200355 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:59.200338 2582 scope.go:117] "RemoveContainer" containerID="1bbd19778f5b97f0c81cc03fc2b7c1c491cec08ee48db103af6e55714a3963f8" Jul 09 14:29:59.200608 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:29:59.200591 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bbd19778f5b97f0c81cc03fc2b7c1c491cec08ee48db103af6e55714a3963f8\": container with ID starting with 1bbd19778f5b97f0c81cc03fc2b7c1c491cec08ee48db103af6e55714a3963f8 not found: ID does not exist" containerID="1bbd19778f5b97f0c81cc03fc2b7c1c491cec08ee48db103af6e55714a3963f8" Jul 09 14:29:59.200647 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:59.200617 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bbd19778f5b97f0c81cc03fc2b7c1c491cec08ee48db103af6e55714a3963f8"} err="failed to get container status \"1bbd19778f5b97f0c81cc03fc2b7c1c491cec08ee48db103af6e55714a3963f8\": rpc error: code = NotFound desc = could not find container \"1bbd19778f5b97f0c81cc03fc2b7c1c491cec08ee48db103af6e55714a3963f8\": container with ID starting with 1bbd19778f5b97f0c81cc03fc2b7c1c491cec08ee48db103af6e55714a3963f8 not found: ID does not exist" Jul 09 14:29:59.212593 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:59.212568 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7"] Jul 09 14:29:59.221127 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:29:59.221104 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kmtr7"] Jul 09 14:30:00.037153 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:00.037119 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50dce30c-480e-4604-8de3-450f22b9a462" path="/var/lib/kubelet/pods/50dce30c-480e-4604-8de3-450f22b9a462/volumes" Jul 09 14:30:19.228655 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.228622 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rfk6g"] Jul 09 14:30:19.229209 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.228987 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50dce30c-480e-4604-8de3-450f22b9a462" containerName="manager" Jul 09 14:30:19.229209 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.229005 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="50dce30c-480e-4604-8de3-450f22b9a462" containerName="manager" Jul 09 14:30:19.229209 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.229097 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="50dce30c-480e-4604-8de3-450f22b9a462" containerName="manager" Jul 09 14:30:19.269688 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.269664 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rfk6g"] Jul 09 14:30:19.269850 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.269756 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" Jul 09 14:30:19.272775 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.272751 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 14:30:19.321750 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.321711 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rfk6g"] Jul 09 14:30:19.355195 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.355165 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89c2f7ab-029e-4e14-87e6-5a283eee81aa-config-file\") pod \"limitador-limitador-64c94fd64f-rfk6g\" (UID: \"89c2f7ab-029e-4e14-87e6-5a283eee81aa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" Jul 09 14:30:19.355347 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.355218 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrkwk\" (UniqueName: \"kubernetes.io/projected/89c2f7ab-029e-4e14-87e6-5a283eee81aa-kube-api-access-qrkwk\") pod \"limitador-limitador-64c94fd64f-rfk6g\" (UID: \"89c2f7ab-029e-4e14-87e6-5a283eee81aa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" Jul 09 14:30:19.455656 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.455617 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrkwk\" (UniqueName: \"kubernetes.io/projected/89c2f7ab-029e-4e14-87e6-5a283eee81aa-kube-api-access-qrkwk\") pod \"limitador-limitador-64c94fd64f-rfk6g\" (UID: \"89c2f7ab-029e-4e14-87e6-5a283eee81aa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" Jul 09 14:30:19.455839 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.455683 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89c2f7ab-029e-4e14-87e6-5a283eee81aa-config-file\") pod \"limitador-limitador-64c94fd64f-rfk6g\" (UID: \"89c2f7ab-029e-4e14-87e6-5a283eee81aa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" Jul 09 14:30:19.456295 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.456273 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89c2f7ab-029e-4e14-87e6-5a283eee81aa-config-file\") pod \"limitador-limitador-64c94fd64f-rfk6g\" (UID: \"89c2f7ab-029e-4e14-87e6-5a283eee81aa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" Jul 09 14:30:19.464764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.464739 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrkwk\" (UniqueName: \"kubernetes.io/projected/89c2f7ab-029e-4e14-87e6-5a283eee81aa-kube-api-access-qrkwk\") pod \"limitador-limitador-64c94fd64f-rfk6g\" (UID: \"89c2f7ab-029e-4e14-87e6-5a283eee81aa\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" Jul 09 14:30:19.580912 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.580856 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" Jul 09 14:30:19.741486 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:19.741462 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rfk6g"] Jul 09 14:30:19.744097 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:30:19.744069 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89c2f7ab_029e_4e14_87e6_5a283eee81aa.slice/crio-a9c6ff2f735e97c2ecfca9ba31f2b736c1f53a14f846a8b221d510cc9bf58568 WatchSource:0}: Error finding container a9c6ff2f735e97c2ecfca9ba31f2b736c1f53a14f846a8b221d510cc9bf58568: Status 404 returned error can't find the container with id a9c6ff2f735e97c2ecfca9ba31f2b736c1f53a14f846a8b221d510cc9bf58568 Jul 09 14:30:20.041501 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:20.041423 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-llx7q"] Jul 09 14:30:20.063177 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:20.063151 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-llx7q"] Jul 09 14:30:20.063317 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:20.063231 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-llx7q" Jul 09 14:30:20.065632 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:20.065611 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mtlgh\"" Jul 09 14:30:20.162182 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:20.162151 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szg2t\" (UniqueName: \"kubernetes.io/projected/c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155-kube-api-access-szg2t\") pod \"authorino-58cd955675-llx7q\" (UID: \"c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155\") " pod="kuadrant-system/authorino-58cd955675-llx7q" Jul 09 14:30:20.263081 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:20.263043 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szg2t\" (UniqueName: \"kubernetes.io/projected/c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155-kube-api-access-szg2t\") pod \"authorino-58cd955675-llx7q\" (UID: \"c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155\") " pod="kuadrant-system/authorino-58cd955675-llx7q" Jul 09 14:30:20.273153 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:20.273127 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szg2t\" (UniqueName: \"kubernetes.io/projected/c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155-kube-api-access-szg2t\") pod \"authorino-58cd955675-llx7q\" (UID: \"c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155\") " pod="kuadrant-system/authorino-58cd955675-llx7q" Jul 09 14:30:20.284751 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:20.284717 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" event={"ID":"89c2f7ab-029e-4e14-87e6-5a283eee81aa","Type":"ContainerStarted","Data":"a9c6ff2f735e97c2ecfca9ba31f2b736c1f53a14f846a8b221d510cc9bf58568"} Jul 09 14:30:20.372256 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:20.372222 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-llx7q" Jul 09 14:30:20.527045 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:20.527005 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-llx7q"] Jul 09 14:30:20.530671 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:30:20.530644 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc39ddaef_6ccd_4fba_9cb6_f6e0fa60f155.slice/crio-a6c51d025f90a308573d368ee5cfc2dd6663c60fd94642c297b8f468a012675e WatchSource:0}: Error finding container a6c51d025f90a308573d368ee5cfc2dd6663c60fd94642c297b8f468a012675e: Status 404 returned error can't find the container with id a6c51d025f90a308573d368ee5cfc2dd6663c60fd94642c297b8f468a012675e Jul 09 14:30:21.306259 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:21.306205 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-llx7q" event={"ID":"c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155","Type":"ContainerStarted","Data":"a6c51d025f90a308573d368ee5cfc2dd6663c60fd94642c297b8f468a012675e"} Jul 09 14:30:25.193890 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:25.193837 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-llx7q"] Jul 09 14:30:25.328781 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:25.328740 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" event={"ID":"89c2f7ab-029e-4e14-87e6-5a283eee81aa","Type":"ContainerStarted","Data":"4babdf4982783953e02d007820ea402d6f8ddabc73cbd024326bd8d2d7f5060b"} Jul 09 14:30:25.328984 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:25.328896 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" Jul 09 14:30:25.330277 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:25.330245 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-llx7q" event={"ID":"c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155","Type":"ContainerStarted","Data":"34da0b9739674c7bfd11eb7f3756c033c82c26044bccc1d37aad832f1273e66e"} Jul 09 14:30:25.345904 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:25.345835 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" podStartSLOduration=1.563449555 podStartE2EDuration="6.345821117s" totalImagesPullingTime="4.782371562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:30:25.344462989 +0000 UTC m=+575.710557374" watchObservedRunningTime="2026-07-09 14:30:25.345821117 +0000 UTC m=+575.711915504" Jul 09 14:30:25.359543 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:25.359501 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-llx7q" podStartSLOduration=1.355324158 podStartE2EDuration="5.359487014s" totalImagesPullingTime="4.004162856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:30:25.35818358 +0000 UTC m=+575.724277979" watchObservedRunningTime="2026-07-09 14:30:25.359487014 +0000 UTC m=+575.725581440" Jul 09 14:30:26.334761 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:26.334714 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-llx7q" podUID="c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155" containerName="authorino" containerID="cri-o://34da0b9739674c7bfd11eb7f3756c033c82c26044bccc1d37aad832f1273e66e" gracePeriod=30 Jul 09 14:30:26.578670 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:26.578646 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-llx7q" Jul 09 14:30:26.718675 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:26.718591 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szg2t\" (UniqueName: \"kubernetes.io/projected/c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155-kube-api-access-szg2t\") pod \"c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155\" (UID: \"c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155\") " Jul 09 14:30:26.720698 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:26.720672 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155-kube-api-access-szg2t" (OuterVolumeSpecName: "kube-api-access-szg2t") pod "c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155" (UID: "c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155"). InnerVolumeSpecName "kube-api-access-szg2t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:30:26.819270 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:26.819239 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szg2t\" (UniqueName: \"kubernetes.io/projected/c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155-kube-api-access-szg2t\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:30:27.340147 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:27.340112 2582 generic.go:356] "Generic (PLEG): container finished" podID="c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155" containerID="34da0b9739674c7bfd11eb7f3756c033c82c26044bccc1d37aad832f1273e66e" exitCode=0 Jul 09 14:30:27.340558 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:27.340151 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-llx7q" Jul 09 14:30:27.340558 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:27.340179 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-llx7q" event={"ID":"c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155","Type":"ContainerDied","Data":"34da0b9739674c7bfd11eb7f3756c033c82c26044bccc1d37aad832f1273e66e"} Jul 09 14:30:27.340558 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:27.340218 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-llx7q" event={"ID":"c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155","Type":"ContainerDied","Data":"a6c51d025f90a308573d368ee5cfc2dd6663c60fd94642c297b8f468a012675e"} Jul 09 14:30:27.340558 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:27.340237 2582 scope.go:117] "RemoveContainer" containerID="34da0b9739674c7bfd11eb7f3756c033c82c26044bccc1d37aad832f1273e66e" Jul 09 14:30:27.350552 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:27.350241 2582 scope.go:117] "RemoveContainer" containerID="34da0b9739674c7bfd11eb7f3756c033c82c26044bccc1d37aad832f1273e66e" Jul 09 14:30:27.351148 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:30:27.351125 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34da0b9739674c7bfd11eb7f3756c033c82c26044bccc1d37aad832f1273e66e\": container with ID starting with 34da0b9739674c7bfd11eb7f3756c033c82c26044bccc1d37aad832f1273e66e not found: ID does not exist" containerID="34da0b9739674c7bfd11eb7f3756c033c82c26044bccc1d37aad832f1273e66e" Jul 09 14:30:27.351277 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:27.351153 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34da0b9739674c7bfd11eb7f3756c033c82c26044bccc1d37aad832f1273e66e"} err="failed to get container status \"34da0b9739674c7bfd11eb7f3756c033c82c26044bccc1d37aad832f1273e66e\": rpc error: code = NotFound desc = could not find container \"34da0b9739674c7bfd11eb7f3756c033c82c26044bccc1d37aad832f1273e66e\": container with ID starting with 34da0b9739674c7bfd11eb7f3756c033c82c26044bccc1d37aad832f1273e66e not found: ID does not exist" Jul 09 14:30:27.361053 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:27.361033 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-llx7q"] Jul 09 14:30:27.364429 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:27.364408 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-llx7q"] Jul 09 14:30:28.038177 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:28.038144 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155" path="/var/lib/kubelet/pods/c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155/volumes" Jul 09 14:30:36.203754 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:36.203714 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rfk6g"] Jul 09 14:30:36.204254 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:36.203954 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" podUID="89c2f7ab-029e-4e14-87e6-5a283eee81aa" containerName="limitador" containerID="cri-o://4babdf4982783953e02d007820ea402d6f8ddabc73cbd024326bd8d2d7f5060b" gracePeriod=30 Jul 09 14:30:36.205389 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:36.204985 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" Jul 09 14:30:36.757531 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:36.757509 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" Jul 09 14:30:36.795277 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:36.795203 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrkwk\" (UniqueName: \"kubernetes.io/projected/89c2f7ab-029e-4e14-87e6-5a283eee81aa-kube-api-access-qrkwk\") pod \"89c2f7ab-029e-4e14-87e6-5a283eee81aa\" (UID: \"89c2f7ab-029e-4e14-87e6-5a283eee81aa\") " Jul 09 14:30:36.795435 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:36.795286 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89c2f7ab-029e-4e14-87e6-5a283eee81aa-config-file\") pod \"89c2f7ab-029e-4e14-87e6-5a283eee81aa\" (UID: \"89c2f7ab-029e-4e14-87e6-5a283eee81aa\") " Jul 09 14:30:36.795651 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:36.795627 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89c2f7ab-029e-4e14-87e6-5a283eee81aa-config-file" (OuterVolumeSpecName: "config-file") pod "89c2f7ab-029e-4e14-87e6-5a283eee81aa" (UID: "89c2f7ab-029e-4e14-87e6-5a283eee81aa"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:30:36.797489 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:36.797464 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89c2f7ab-029e-4e14-87e6-5a283eee81aa-kube-api-access-qrkwk" (OuterVolumeSpecName: "kube-api-access-qrkwk") pod "89c2f7ab-029e-4e14-87e6-5a283eee81aa" (UID: "89c2f7ab-029e-4e14-87e6-5a283eee81aa"). InnerVolumeSpecName "kube-api-access-qrkwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:30:36.895963 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:36.895930 2582 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89c2f7ab-029e-4e14-87e6-5a283eee81aa-config-file\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:30:36.895963 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:36.895959 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrkwk\" (UniqueName: \"kubernetes.io/projected/89c2f7ab-029e-4e14-87e6-5a283eee81aa-kube-api-access-qrkwk\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:30:37.385426 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:37.385388 2582 generic.go:356] "Generic (PLEG): container finished" podID="89c2f7ab-029e-4e14-87e6-5a283eee81aa" containerID="4babdf4982783953e02d007820ea402d6f8ddabc73cbd024326bd8d2d7f5060b" exitCode=0 Jul 09 14:30:37.385846 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:37.385448 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" Jul 09 14:30:37.385846 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:37.385479 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" event={"ID":"89c2f7ab-029e-4e14-87e6-5a283eee81aa","Type":"ContainerDied","Data":"4babdf4982783953e02d007820ea402d6f8ddabc73cbd024326bd8d2d7f5060b"} Jul 09 14:30:37.385846 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:37.385532 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rfk6g" event={"ID":"89c2f7ab-029e-4e14-87e6-5a283eee81aa","Type":"ContainerDied","Data":"a9c6ff2f735e97c2ecfca9ba31f2b736c1f53a14f846a8b221d510cc9bf58568"} Jul 09 14:30:37.385846 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:37.385555 2582 scope.go:117] "RemoveContainer" containerID="4babdf4982783953e02d007820ea402d6f8ddabc73cbd024326bd8d2d7f5060b" Jul 09 14:30:37.393955 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:37.393937 2582 scope.go:117] "RemoveContainer" containerID="4babdf4982783953e02d007820ea402d6f8ddabc73cbd024326bd8d2d7f5060b" Jul 09 14:30:37.394184 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:30:37.394168 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4babdf4982783953e02d007820ea402d6f8ddabc73cbd024326bd8d2d7f5060b\": container with ID starting with 4babdf4982783953e02d007820ea402d6f8ddabc73cbd024326bd8d2d7f5060b not found: ID does not exist" containerID="4babdf4982783953e02d007820ea402d6f8ddabc73cbd024326bd8d2d7f5060b" Jul 09 14:30:37.394255 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:37.394192 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4babdf4982783953e02d007820ea402d6f8ddabc73cbd024326bd8d2d7f5060b"} err="failed to get container status \"4babdf4982783953e02d007820ea402d6f8ddabc73cbd024326bd8d2d7f5060b\": rpc error: code = NotFound desc = could not find container \"4babdf4982783953e02d007820ea402d6f8ddabc73cbd024326bd8d2d7f5060b\": container with ID starting with 4babdf4982783953e02d007820ea402d6f8ddabc73cbd024326bd8d2d7f5060b not found: ID does not exist" Jul 09 14:30:37.406995 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:37.406970 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rfk6g"] Jul 09 14:30:37.411054 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:37.411027 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rfk6g"] Jul 09 14:30:38.037131 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:38.037099 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89c2f7ab-029e-4e14-87e6-5a283eee81aa" path="/var/lib/kubelet/pods/89c2f7ab-029e-4e14-87e6-5a283eee81aa/volumes" Jul 09 14:30:41.016834 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.016803 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg"] Jul 09 14:30:41.017411 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.017386 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89c2f7ab-029e-4e14-87e6-5a283eee81aa" containerName="limitador" Jul 09 14:30:41.017501 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.017414 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="89c2f7ab-029e-4e14-87e6-5a283eee81aa" containerName="limitador" Jul 09 14:30:41.017501 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.017452 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155" containerName="authorino" Jul 09 14:30:41.017501 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.017462 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155" containerName="authorino" Jul 09 14:30:41.017640 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.017536 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c39ddaef-6ccd-4fba-9cb6-f6e0fa60f155" containerName="authorino" Jul 09 14:30:41.017640 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.017549 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="89c2f7ab-029e-4e14-87e6-5a283eee81aa" containerName="limitador" Jul 09 14:30:41.021610 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.021589 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg" Jul 09 14:30:41.024354 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.024330 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 09 14:30:41.024471 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.024339 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 14:30:41.024471 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.024398 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-mn4c5\"" Jul 09 14:30:41.025644 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.025626 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 14:30:41.029374 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.029354 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg"] Jul 09 14:30:41.133009 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.132975 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7rnh\" (UniqueName: \"kubernetes.io/projected/c9a5b486-912e-4fc7-b78d-1016b0040365-kube-api-access-b7rnh\") pod \"postgres-69fb4594cf-lfzsg\" (UID: \"c9a5b486-912e-4fc7-b78d-1016b0040365\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg" Jul 09 14:30:41.133180 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.133036 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c9a5b486-912e-4fc7-b78d-1016b0040365-data\") pod \"postgres-69fb4594cf-lfzsg\" (UID: \"c9a5b486-912e-4fc7-b78d-1016b0040365\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg" Jul 09 14:30:41.234085 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.234051 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7rnh\" (UniqueName: \"kubernetes.io/projected/c9a5b486-912e-4fc7-b78d-1016b0040365-kube-api-access-b7rnh\") pod \"postgres-69fb4594cf-lfzsg\" (UID: \"c9a5b486-912e-4fc7-b78d-1016b0040365\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg" Jul 09 14:30:41.234249 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.234100 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c9a5b486-912e-4fc7-b78d-1016b0040365-data\") pod \"postgres-69fb4594cf-lfzsg\" (UID: \"c9a5b486-912e-4fc7-b78d-1016b0040365\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg" Jul 09 14:30:41.234444 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.234428 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c9a5b486-912e-4fc7-b78d-1016b0040365-data\") pod \"postgres-69fb4594cf-lfzsg\" (UID: \"c9a5b486-912e-4fc7-b78d-1016b0040365\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg" Jul 09 14:30:41.243115 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.243083 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7rnh\" (UniqueName: \"kubernetes.io/projected/c9a5b486-912e-4fc7-b78d-1016b0040365-kube-api-access-b7rnh\") pod \"postgres-69fb4594cf-lfzsg\" (UID: \"c9a5b486-912e-4fc7-b78d-1016b0040365\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg" Jul 09 14:30:41.332799 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.332770 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg" Jul 09 14:30:41.466806 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:41.466774 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg"] Jul 09 14:30:41.471147 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:30:41.471102 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a5b486_912e_4fc7_b78d_1016b0040365.slice/crio-be3762669fe2072d7b40f9f973ef5843a368ea4a3bac124a4bd824226b3d0b59 WatchSource:0}: Error finding container be3762669fe2072d7b40f9f973ef5843a368ea4a3bac124a4bd824226b3d0b59: Status 404 returned error can't find the container with id be3762669fe2072d7b40f9f973ef5843a368ea4a3bac124a4bd824226b3d0b59 Jul 09 14:30:42.412233 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:42.412191 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg" event={"ID":"c9a5b486-912e-4fc7-b78d-1016b0040365","Type":"ContainerStarted","Data":"be3762669fe2072d7b40f9f973ef5843a368ea4a3bac124a4bd824226b3d0b59"} Jul 09 14:30:46.433411 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:46.433314 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg" event={"ID":"c9a5b486-912e-4fc7-b78d-1016b0040365","Type":"ContainerStarted","Data":"ce654d505be9444a81293ccdbfe71471e7544eb61715b5ed9f603c08689fdabe"} Jul 09 14:30:46.433411 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:46.433387 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg" Jul 09 14:30:46.451164 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:46.451096 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg" podStartSLOduration=1.847444904 podStartE2EDuration="6.451078563s" totalImagesPullingTime="4.603633659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:30:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:30:46.450242625 +0000 UTC m=+596.816337011" watchObservedRunningTime="2026-07-09 14:30:46.451078563 +0000 UTC m=+596.817172949" Jul 09 14:30:52.467717 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:30:52.467687 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lfzsg" Jul 09 14:31:35.917767 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:35.917734 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-xhrkm"] Jul 09 14:31:35.921048 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:35.921032 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-xhrkm" Jul 09 14:31:35.926402 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:35.926380 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 14:31:35.937177 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:35.937153 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-xhrkm"] Jul 09 14:31:35.992500 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:35.992463 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-stddl"] Jul 09 14:31:35.995708 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:35.995687 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-stddl" Jul 09 14:31:35.998391 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:35.998367 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/633c7032-c50e-4182-bd2d-fee0d920768e-plugins-config-volume\") pod \"payload-processing-58f86c7798-xhrkm\" (UID: \"633c7032-c50e-4182-bd2d-fee0d920768e\") " pod="openshift-ingress/payload-processing-58f86c7798-xhrkm" Jul 09 14:31:35.998496 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:35.998419 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx7dm\" (UniqueName: \"kubernetes.io/projected/633c7032-c50e-4182-bd2d-fee0d920768e-kube-api-access-zx7dm\") pod \"payload-processing-58f86c7798-xhrkm\" (UID: \"633c7032-c50e-4182-bd2d-fee0d920768e\") " pod="openshift-ingress/payload-processing-58f86c7798-xhrkm" Jul 09 14:31:35.999619 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:35.999591 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-6r4vh\"" Jul 09 14:31:36.011920 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.011899 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-stddl"] Jul 09 14:31:36.099834 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.099799 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx7dm\" (UniqueName: \"kubernetes.io/projected/633c7032-c50e-4182-bd2d-fee0d920768e-kube-api-access-zx7dm\") pod \"payload-processing-58f86c7798-xhrkm\" (UID: \"633c7032-c50e-4182-bd2d-fee0d920768e\") " pod="openshift-ingress/payload-processing-58f86c7798-xhrkm" Jul 09 14:31:36.100007 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.099858 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfhht\" (UniqueName: \"kubernetes.io/projected/4cdb3207-2563-400d-9dcc-043597f8e0e0-kube-api-access-tfhht\") pod \"payload-pre-processing-7964457696-stddl\" (UID: \"4cdb3207-2563-400d-9dcc-043597f8e0e0\") " pod="openshift-ingress/payload-pre-processing-7964457696-stddl" Jul 09 14:31:36.100007 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.099912 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cdb3207-2563-400d-9dcc-043597f8e0e0-plugins-config-volume\") pod \"payload-pre-processing-7964457696-stddl\" (UID: \"4cdb3207-2563-400d-9dcc-043597f8e0e0\") " pod="openshift-ingress/payload-pre-processing-7964457696-stddl" Jul 09 14:31:36.100007 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.099934 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/633c7032-c50e-4182-bd2d-fee0d920768e-plugins-config-volume\") pod \"payload-processing-58f86c7798-xhrkm\" (UID: \"633c7032-c50e-4182-bd2d-fee0d920768e\") " pod="openshift-ingress/payload-processing-58f86c7798-xhrkm" Jul 09 14:31:36.100646 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.100626 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/633c7032-c50e-4182-bd2d-fee0d920768e-plugins-config-volume\") pod \"payload-processing-58f86c7798-xhrkm\" (UID: \"633c7032-c50e-4182-bd2d-fee0d920768e\") " pod="openshift-ingress/payload-processing-58f86c7798-xhrkm" Jul 09 14:31:36.108759 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.108728 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx7dm\" (UniqueName: \"kubernetes.io/projected/633c7032-c50e-4182-bd2d-fee0d920768e-kube-api-access-zx7dm\") pod \"payload-processing-58f86c7798-xhrkm\" (UID: \"633c7032-c50e-4182-bd2d-fee0d920768e\") " pod="openshift-ingress/payload-processing-58f86c7798-xhrkm" Jul 09 14:31:36.200667 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.200586 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfhht\" (UniqueName: \"kubernetes.io/projected/4cdb3207-2563-400d-9dcc-043597f8e0e0-kube-api-access-tfhht\") pod \"payload-pre-processing-7964457696-stddl\" (UID: \"4cdb3207-2563-400d-9dcc-043597f8e0e0\") " pod="openshift-ingress/payload-pre-processing-7964457696-stddl" Jul 09 14:31:36.200667 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.200633 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cdb3207-2563-400d-9dcc-043597f8e0e0-plugins-config-volume\") pod \"payload-pre-processing-7964457696-stddl\" (UID: \"4cdb3207-2563-400d-9dcc-043597f8e0e0\") " pod="openshift-ingress/payload-pre-processing-7964457696-stddl" Jul 09 14:31:36.201405 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.201387 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cdb3207-2563-400d-9dcc-043597f8e0e0-plugins-config-volume\") pod \"payload-pre-processing-7964457696-stddl\" (UID: \"4cdb3207-2563-400d-9dcc-043597f8e0e0\") " pod="openshift-ingress/payload-pre-processing-7964457696-stddl" Jul 09 14:31:36.209825 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.209802 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfhht\" (UniqueName: \"kubernetes.io/projected/4cdb3207-2563-400d-9dcc-043597f8e0e0-kube-api-access-tfhht\") pod \"payload-pre-processing-7964457696-stddl\" (UID: \"4cdb3207-2563-400d-9dcc-043597f8e0e0\") " pod="openshift-ingress/payload-pre-processing-7964457696-stddl" Jul 09 14:31:36.230662 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.230638 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-xhrkm" Jul 09 14:31:36.306474 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.306427 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-stddl" Jul 09 14:31:36.381904 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.381860 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-xhrkm"] Jul 09 14:31:36.385268 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:31:36.385241 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod633c7032_c50e_4182_bd2d_fee0d920768e.slice/crio-e653970f5f7a655d4f7177b44538af9f552fe24d839017c1f3bf486900488384 WatchSource:0}: Error finding container e653970f5f7a655d4f7177b44538af9f552fe24d839017c1f3bf486900488384: Status 404 returned error can't find the container with id e653970f5f7a655d4f7177b44538af9f552fe24d839017c1f3bf486900488384 Jul 09 14:31:36.453324 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.453242 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-stddl"] Jul 09 14:31:36.457419 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:31:36.457382 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cdb3207_2563_400d_9dcc_043597f8e0e0.slice/crio-b53481865c9c288768410cc482e5f910931e8fedc0a09463789b74b31cf11805 WatchSource:0}: Error finding container b53481865c9c288768410cc482e5f910931e8fedc0a09463789b74b31cf11805: Status 404 returned error can't find the container with id b53481865c9c288768410cc482e5f910931e8fedc0a09463789b74b31cf11805 Jul 09 14:31:36.650182 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.650145 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-xhrkm" event={"ID":"633c7032-c50e-4182-bd2d-fee0d920768e","Type":"ContainerStarted","Data":"e653970f5f7a655d4f7177b44538af9f552fe24d839017c1f3bf486900488384"} Jul 09 14:31:36.651471 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:36.651440 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-stddl" event={"ID":"4cdb3207-2563-400d-9dcc-043597f8e0e0","Type":"ContainerStarted","Data":"b53481865c9c288768410cc482e5f910931e8fedc0a09463789b74b31cf11805"} Jul 09 14:31:38.661969 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:38.661907 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-stddl" event={"ID":"4cdb3207-2563-400d-9dcc-043597f8e0e0","Type":"ContainerStarted","Data":"82e1e0eb7fb14ef4e3ee49ae042ed7588674eabaa9d4692bb33cafc134e7ae23"} Jul 09 14:31:38.662449 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:38.662069 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-stddl" Jul 09 14:31:38.663463 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:38.663438 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-xhrkm" event={"ID":"633c7032-c50e-4182-bd2d-fee0d920768e","Type":"ContainerStarted","Data":"332a9d816fd17abb849f82c0d4e84b710dce32ceb497c59e7f05cf9e583881ec"} Jul 09 14:31:38.663574 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:38.663562 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-xhrkm" Jul 09 14:31:38.685744 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:38.685700 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-stddl" podStartSLOduration=2.276054499 podStartE2EDuration="3.685683274s" totalImagesPullingTime="1.409628775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:31:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:31:38.684153172 +0000 UTC m=+649.050247560" watchObservedRunningTime="2026-07-09 14:31:38.685683274 +0000 UTC m=+649.051777661" Jul 09 14:31:38.710016 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:38.709976 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-xhrkm" podStartSLOduration=2.250125926 podStartE2EDuration="3.709964497s" totalImagesPullingTime="1.459838571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:31:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:31:38.708644505 +0000 UTC m=+649.074738904" watchObservedRunningTime="2026-07-09 14:31:38.709964497 +0000 UTC m=+649.076058883" Jul 09 14:31:49.669749 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:49.669716 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-xhrkm" Jul 09 14:31:49.670237 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:31:49.669785 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-stddl" Jul 09 14:32:16.498559 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.498528 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg"] Jul 09 14:32:16.500816 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.500793 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.504488 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.504459 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 14:32:16.504620 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.504462 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ndfhx\"" Jul 09 14:32:16.504620 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.504503 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 09 14:32:16.504620 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.504517 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 14:32:16.511541 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.511517 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg"] Jul 09 14:32:16.552350 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.552312 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.552350 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.552348 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htnsf\" (UniqueName: \"kubernetes.io/projected/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-kube-api-access-htnsf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.552600 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.552464 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.552600 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.552514 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.552600 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.552537 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.552600 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.552570 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.653395 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.653363 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.653576 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.653398 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htnsf\" (UniqueName: \"kubernetes.io/projected/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-kube-api-access-htnsf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.653576 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.653448 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.653576 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.653468 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.653576 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.653482 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.653576 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.653502 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.653976 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.653949 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.654076 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.653991 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.654076 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.654011 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.656003 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.655976 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.656132 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.656115 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.662239 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.662216 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htnsf\" (UniqueName: \"kubernetes.io/projected/bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb-kube-api-access-htnsf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6zcjg\" (UID: \"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.812164 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.812136 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:16.944892 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.944831 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg"] Jul 09 14:32:16.947683 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:32:16.947653 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbffaa6e8_e33e_4493_aeb2_973eb6ab5ffb.slice/crio-a9486215526ad17c87fd62c00a1ad1c694f323edd30f18f39f969a2d5bbd0541 WatchSource:0}: Error finding container a9486215526ad17c87fd62c00a1ad1c694f323edd30f18f39f969a2d5bbd0541: Status 404 returned error can't find the container with id a9486215526ad17c87fd62c00a1ad1c694f323edd30f18f39f969a2d5bbd0541 Jul 09 14:32:16.949242 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:16.949214 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 14:32:17.832987 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:17.832945 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" event={"ID":"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb","Type":"ContainerStarted","Data":"a9486215526ad17c87fd62c00a1ad1c694f323edd30f18f39f969a2d5bbd0541"} Jul 09 14:32:18.839531 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:18.839487 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" event={"ID":"bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb","Type":"ContainerStarted","Data":"60fbb85cce43861af8c948544e0db47ca8f8e70240c957fe65270c3191d37aec"} Jul 09 14:32:18.858931 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:18.858860 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" podStartSLOduration=1.758268881 podStartE2EDuration="2.85884449s" totalImagesPullingTime="1.100575609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:32:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:32:18.858496842 +0000 UTC m=+689.224591230" watchObservedRunningTime="2026-07-09 14:32:18.85884449 +0000 UTC m=+689.224938876" Jul 09 14:32:26.812508 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:26.812464 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:26.813084 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:26.812533 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:26.824956 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:26.824933 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:26.885321 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:26.885290 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6zcjg" Jul 09 14:32:27.907591 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:27.907560 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc"] Jul 09 14:32:27.909944 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:27.909927 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:27.912657 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:27.912637 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 09 14:32:27.925030 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:27.925009 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc"] Jul 09 14:32:27.950312 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:27.950287 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:27.950462 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:27.950323 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:27.950462 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:27.950353 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:27.950462 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:27.950398 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:27.950462 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:27.950431 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:27.950462 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:27.950452 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lws2c\" (UniqueName: \"kubernetes.io/projected/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-kube-api-access-lws2c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:28.051536 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.051501 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:28.051705 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.051545 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:28.051705 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.051582 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:28.051705 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.051612 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:28.051705 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.051640 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:28.051705 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.051667 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lws2c\" (UniqueName: \"kubernetes.io/projected/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-kube-api-access-lws2c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:28.052134 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.052107 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:28.052260 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.052215 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:28.052438 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.052404 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:28.054534 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.054500 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:28.054693 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.054679 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:28.061970 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.061945 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lws2c\" (UniqueName: \"kubernetes.io/projected/0a8eab4b-0985-4366-8a9c-2d8c551fc29c-kube-api-access-lws2c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc\" (UID: \"0a8eab4b-0985-4366-8a9c-2d8c551fc29c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:28.219835 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.219753 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:28.356394 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.356369 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc"] Jul 09 14:32:28.358383 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:32:28.358356 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a8eab4b_0985_4366_8a9c_2d8c551fc29c.slice/crio-2252fa5ae5d463cbe83d3d44d47eaef13a0e1d33dd355f5a0093e3eedf74883a WatchSource:0}: Error finding container 2252fa5ae5d463cbe83d3d44d47eaef13a0e1d33dd355f5a0093e3eedf74883a: Status 404 returned error can't find the container with id 2252fa5ae5d463cbe83d3d44d47eaef13a0e1d33dd355f5a0093e3eedf74883a Jul 09 14:32:28.884593 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.884504 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" event={"ID":"0a8eab4b-0985-4366-8a9c-2d8c551fc29c","Type":"ContainerStarted","Data":"60239c463b21440e851c3e1bea7c7514986115542ac9105a5a3df58a1fe7259c"} Jul 09 14:32:28.884593 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.884551 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" event={"ID":"0a8eab4b-0985-4366-8a9c-2d8c551fc29c","Type":"ContainerStarted","Data":"2252fa5ae5d463cbe83d3d44d47eaef13a0e1d33dd355f5a0093e3eedf74883a"} Jul 09 14:32:28.914321 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:28.914270 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" podStartSLOduration=1.682745235 podStartE2EDuration="1.914254086s" totalImagesPullingTime="231.508851ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:32:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:32:28.912678968 +0000 UTC m=+699.278773367" watchObservedRunningTime="2026-07-09 14:32:28.914254086 +0000 UTC m=+699.280348471" Jul 09 14:32:38.219886 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:38.219839 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:38.220265 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:38.219940 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:38.232518 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:38.232489 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:38.940963 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:38.940932 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc" Jul 09 14:32:39.503686 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.503638 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc"] Jul 09 14:32:39.510274 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.510256 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.512733 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.512717 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 09 14:32:39.517297 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.517272 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc"] Jul 09 14:32:39.657698 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.657660 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g767\" (UniqueName: \"kubernetes.io/projected/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-kube-api-access-4g767\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.657863 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.657732 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.657863 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.657759 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.657863 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.657777 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.657863 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.657797 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.657863 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.657858 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.758522 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.758438 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.758522 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.758479 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.758522 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.758499 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.758772 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.758625 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.758772 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.758685 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.758772 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.758743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g767\" (UniqueName: \"kubernetes.io/projected/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-kube-api-access-4g767\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.758978 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.758849 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.758978 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.758898 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.759070 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.758996 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.760933 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.760911 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.761224 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.761204 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.767395 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.767375 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g767\" (UniqueName: \"kubernetes.io/projected/6f4278e9-b4c4-4c63-9433-fb5e012e1c23-kube-api-access-4g767\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-49kpc\" (UID: \"6f4278e9-b4c4-4c63-9433-fb5e012e1c23\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.821455 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.821419 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:39.957434 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:39.957401 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc"] Jul 09 14:32:39.960573 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:32:39.960546 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f4278e9_b4c4_4c63_9433_fb5e012e1c23.slice/crio-9814b6ab26bf9ba15b517f328c6c2d955d17af42abccf693937040cc7640b3a0 WatchSource:0}: Error finding container 9814b6ab26bf9ba15b517f328c6c2d955d17af42abccf693937040cc7640b3a0: Status 404 returned error can't find the container with id 9814b6ab26bf9ba15b517f328c6c2d955d17af42abccf693937040cc7640b3a0 Jul 09 14:32:40.941066 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:40.941030 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" event={"ID":"6f4278e9-b4c4-4c63-9433-fb5e012e1c23","Type":"ContainerStarted","Data":"2743a34f7d8eaaf89ff2d713bb741a5ec1f143c8e54c54d2f962fc81e53d80a5"} Jul 09 14:32:40.941066 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:40.941093 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" event={"ID":"6f4278e9-b4c4-4c63-9433-fb5e012e1c23","Type":"ContainerStarted","Data":"9814b6ab26bf9ba15b517f328c6c2d955d17af42abccf693937040cc7640b3a0"} Jul 09 14:32:40.970632 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:40.970418 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" podStartSLOduration=1.742646713 podStartE2EDuration="1.97039775s" totalImagesPullingTime="227.751037ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:32:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:32:40.968651254 +0000 UTC m=+711.334745653" watchObservedRunningTime="2026-07-09 14:32:40.97039775 +0000 UTC m=+711.336492139" Jul 09 14:32:49.822346 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:49.822309 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:49.822798 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:49.822400 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:49.834741 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:49.834716 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:32:49.991020 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:32:49.990988 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-49kpc" Jul 09 14:39:55.404451 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:55.404415 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z"] Jul 09 14:39:55.408113 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:55.404638 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" podUID="a14878ce-f095-4066-993c-daad867c5d44" containerName="manager" containerID="cri-o://8a3978466292a2ca01c9aef25a4c981edc404a55c6c3abd1a217c8a7403b3ea1" gracePeriod=10 Jul 09 14:39:58.144249 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.144208 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" podUID="a14878ce-f095-4066-993c-daad867c5d44" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.31:8081/readyz\": dial tcp 10.133.0.31:8081: connect: connection refused" Jul 09 14:39:58.260733 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.260707 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" Jul 09 14:39:58.306771 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.306691 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a14878ce-f095-4066-993c-daad867c5d44-extensions-socket-volume\") pod \"a14878ce-f095-4066-993c-daad867c5d44\" (UID: \"a14878ce-f095-4066-993c-daad867c5d44\") " Jul 09 14:39:58.306771 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.306766 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppzv5\" (UniqueName: \"kubernetes.io/projected/a14878ce-f095-4066-993c-daad867c5d44-kube-api-access-ppzv5\") pod \"a14878ce-f095-4066-993c-daad867c5d44\" (UID: \"a14878ce-f095-4066-993c-daad867c5d44\") " Jul 09 14:39:58.307148 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.307115 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a14878ce-f095-4066-993c-daad867c5d44-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a14878ce-f095-4066-993c-daad867c5d44" (UID: "a14878ce-f095-4066-993c-daad867c5d44"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:39:58.309065 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.309038 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a14878ce-f095-4066-993c-daad867c5d44-kube-api-access-ppzv5" (OuterVolumeSpecName: "kube-api-access-ppzv5") pod "a14878ce-f095-4066-993c-daad867c5d44" (UID: "a14878ce-f095-4066-993c-daad867c5d44"). InnerVolumeSpecName "kube-api-access-ppzv5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:39:58.407819 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.407787 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a14878ce-f095-4066-993c-daad867c5d44-extensions-socket-volume\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:39:58.407819 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.407812 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppzv5\" (UniqueName: \"kubernetes.io/projected/a14878ce-f095-4066-993c-daad867c5d44-kube-api-access-ppzv5\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 14:39:58.917727 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.917690 2582 generic.go:356] "Generic (PLEG): container finished" podID="a14878ce-f095-4066-993c-daad867c5d44" containerID="8a3978466292a2ca01c9aef25a4c981edc404a55c6c3abd1a217c8a7403b3ea1" exitCode=0 Jul 09 14:39:58.917909 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.917732 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" Jul 09 14:39:58.917909 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.917748 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" event={"ID":"a14878ce-f095-4066-993c-daad867c5d44","Type":"ContainerDied","Data":"8a3978466292a2ca01c9aef25a4c981edc404a55c6c3abd1a217c8a7403b3ea1"} Jul 09 14:39:58.917909 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.917803 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z" event={"ID":"a14878ce-f095-4066-993c-daad867c5d44","Type":"ContainerDied","Data":"471162de8ac7a85968ecb29d423c8615e8f89c3b24f61aa34305fdf4be6060ba"} Jul 09 14:39:58.917909 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.917833 2582 scope.go:117] "RemoveContainer" containerID="8a3978466292a2ca01c9aef25a4c981edc404a55c6c3abd1a217c8a7403b3ea1" Jul 09 14:39:58.927011 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.926992 2582 scope.go:117] "RemoveContainer" containerID="8a3978466292a2ca01c9aef25a4c981edc404a55c6c3abd1a217c8a7403b3ea1" Jul 09 14:39:58.927262 ip-10-0-136-93 kubenswrapper[2582]: E0709 14:39:58.927225 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a3978466292a2ca01c9aef25a4c981edc404a55c6c3abd1a217c8a7403b3ea1\": container with ID starting with 8a3978466292a2ca01c9aef25a4c981edc404a55c6c3abd1a217c8a7403b3ea1 not found: ID does not exist" containerID="8a3978466292a2ca01c9aef25a4c981edc404a55c6c3abd1a217c8a7403b3ea1" Jul 09 14:39:58.927309 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.927269 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a3978466292a2ca01c9aef25a4c981edc404a55c6c3abd1a217c8a7403b3ea1"} err="failed to get container status \"8a3978466292a2ca01c9aef25a4c981edc404a55c6c3abd1a217c8a7403b3ea1\": rpc error: code = NotFound desc = could not find container \"8a3978466292a2ca01c9aef25a4c981edc404a55c6c3abd1a217c8a7403b3ea1\": container with ID starting with 8a3978466292a2ca01c9aef25a4c981edc404a55c6c3abd1a217c8a7403b3ea1 not found: ID does not exist" Jul 09 14:39:58.940349 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.940315 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z"] Jul 09 14:39:58.943634 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:39:58.943611 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8s2z"] Jul 09 14:40:00.041659 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:00.041630 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a14878ce-f095-4066-993c-daad867c5d44" path="/var/lib/kubelet/pods/a14878ce-f095-4066-993c-daad867c5d44/volumes" Jul 09 14:40:56.582764 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:56.582726 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb"] Jul 09 14:40:56.583179 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:56.583109 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a14878ce-f095-4066-993c-daad867c5d44" containerName="manager" Jul 09 14:40:56.583179 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:56.583121 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a14878ce-f095-4066-993c-daad867c5d44" containerName="manager" Jul 09 14:40:56.583261 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:56.583190 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a14878ce-f095-4066-993c-daad867c5d44" containerName="manager" Jul 09 14:40:56.586184 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:56.586167 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb" Jul 09 14:40:56.588840 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:56.588819 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hgl5d\"" Jul 09 14:40:56.598562 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:56.598528 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb"] Jul 09 14:40:56.686445 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:56.686412 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k8k4\" (UniqueName: \"kubernetes.io/projected/484d17fb-4c43-4cda-b8a3-51885ad03ab6-kube-api-access-4k8k4\") pod \"kuadrant-operator-controller-manager-86878c769f-6v7fb\" (UID: \"484d17fb-4c43-4cda-b8a3-51885ad03ab6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb" Jul 09 14:40:56.686648 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:56.686495 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/484d17fb-4c43-4cda-b8a3-51885ad03ab6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6v7fb\" (UID: \"484d17fb-4c43-4cda-b8a3-51885ad03ab6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb" Jul 09 14:40:56.787237 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:56.787200 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/484d17fb-4c43-4cda-b8a3-51885ad03ab6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6v7fb\" (UID: \"484d17fb-4c43-4cda-b8a3-51885ad03ab6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb" Jul 09 14:40:56.787412 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:56.787255 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k8k4\" (UniqueName: \"kubernetes.io/projected/484d17fb-4c43-4cda-b8a3-51885ad03ab6-kube-api-access-4k8k4\") pod \"kuadrant-operator-controller-manager-86878c769f-6v7fb\" (UID: \"484d17fb-4c43-4cda-b8a3-51885ad03ab6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb" Jul 09 14:40:56.787582 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:56.787561 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/484d17fb-4c43-4cda-b8a3-51885ad03ab6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6v7fb\" (UID: \"484d17fb-4c43-4cda-b8a3-51885ad03ab6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb" Jul 09 14:40:56.804142 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:56.804115 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k8k4\" (UniqueName: \"kubernetes.io/projected/484d17fb-4c43-4cda-b8a3-51885ad03ab6-kube-api-access-4k8k4\") pod \"kuadrant-operator-controller-manager-86878c769f-6v7fb\" (UID: \"484d17fb-4c43-4cda-b8a3-51885ad03ab6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb" Jul 09 14:40:56.896700 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:56.896628 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb" Jul 09 14:40:57.044425 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:57.044390 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb"] Jul 09 14:40:57.048334 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:40:57.048307 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod484d17fb_4c43_4cda_b8a3_51885ad03ab6.slice/crio-244dbd676633f414fb75a0e6b8b1a6576c3815097d0deeb6a263d82099e2dc02 WatchSource:0}: Error finding container 244dbd676633f414fb75a0e6b8b1a6576c3815097d0deeb6a263d82099e2dc02: Status 404 returned error can't find the container with id 244dbd676633f414fb75a0e6b8b1a6576c3815097d0deeb6a263d82099e2dc02 Jul 09 14:40:57.169676 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:57.169591 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb" event={"ID":"484d17fb-4c43-4cda-b8a3-51885ad03ab6","Type":"ContainerStarted","Data":"dbf3558cf1c4f24df6e28dd849554bd6219a26defeba63cd2a9be90cbe7468ee"} Jul 09 14:40:57.169676 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:57.169648 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb" event={"ID":"484d17fb-4c43-4cda-b8a3-51885ad03ab6","Type":"ContainerStarted","Data":"244dbd676633f414fb75a0e6b8b1a6576c3815097d0deeb6a263d82099e2dc02"} Jul 09 14:40:57.169857 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:57.169720 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb" Jul 09 14:40:57.195641 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:40:57.195597 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb" podStartSLOduration=1.195579845 podStartE2EDuration="1.195579845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:40:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:40:57.193317482 +0000 UTC m=+1207.559411868" watchObservedRunningTime="2026-07-09 14:40:57.195579845 +0000 UTC m=+1207.561674231" Jul 09 14:41:08.183224 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:41:08.183194 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6v7fb" Jul 09 14:59:17.105969 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.105935 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq"] Jul 09 14:59:17.108833 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.108816 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.111517 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.111494 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0d877f\"/\"openshift-service-ca.crt\"" Jul 09 14:59:17.111652 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.111536 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0d877f\"/\"sub-test-model-0d877f-kserve-self-signed-certs\"" Jul 09 14:59:17.112663 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.112648 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0d877f\"/\"kube-root-ca.crt\"" Jul 09 14:59:17.112746 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.112678 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0d877f\"/\"default-dockercfg-gmsw6\"" Jul 09 14:59:17.120200 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.120178 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq"] Jul 09 14:59:17.272522 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.272482 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2157287-82ef-4364-87e0-e7fd8d62d1c3-tls-certs\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.272687 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.272532 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-home\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.272687 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.272614 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-model-cache\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.272687 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.272645 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-tmp-dir\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.272799 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.272690 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-824fg\" (UniqueName: \"kubernetes.io/projected/b2157287-82ef-4364-87e0-e7fd8d62d1c3-kube-api-access-824fg\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.272799 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.272722 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-dshm\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.373907 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.373787 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-model-cache\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.373907 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.373825 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-tmp-dir\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.374135 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.373921 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-824fg\" (UniqueName: \"kubernetes.io/projected/b2157287-82ef-4364-87e0-e7fd8d62d1c3-kube-api-access-824fg\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.374135 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.373955 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-dshm\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.374135 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.373983 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2157287-82ef-4364-87e0-e7fd8d62d1c3-tls-certs\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.374135 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.374018 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-home\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.374335 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.374273 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-model-cache\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.374335 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.374284 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-tmp-dir\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.374473 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.374453 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-home\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.376445 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.376418 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-dshm\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.376529 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.376512 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2157287-82ef-4364-87e0-e7fd8d62d1c3-tls-certs\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.381699 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.381679 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-824fg\" (UniqueName: \"kubernetes.io/projected/b2157287-82ef-4364-87e0-e7fd8d62d1c3-kube-api-access-824fg\") pod \"sub-test-model-0d877f-kserve-5946fb6555-g65qq\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.418858 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.418817 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:17.556615 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:17.556589 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq"] Jul 09 14:59:17.558616 ip-10-0-136-93 kubenswrapper[2582]: W0709 14:59:17.558573 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2157287_82ef_4364_87e0_e7fd8d62d1c3.slice/crio-05d3165809e25917d7ad5006fe339513056e388677fe147892e2b0363519660d WatchSource:0}: Error finding container 05d3165809e25917d7ad5006fe339513056e388677fe147892e2b0363519660d: Status 404 returned error can't find the container with id 05d3165809e25917d7ad5006fe339513056e388677fe147892e2b0363519660d Jul 09 14:59:18.195466 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:18.195421 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" event={"ID":"b2157287-82ef-4364-87e0-e7fd8d62d1c3","Type":"ContainerStarted","Data":"fc826f3b74dfb62f5766623af90eed9c79de9619261a8e2dd78903e10e7dcfeb"} Jul 09 14:59:18.195842 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:18.195474 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" event={"ID":"b2157287-82ef-4364-87e0-e7fd8d62d1c3","Type":"ContainerStarted","Data":"05d3165809e25917d7ad5006fe339513056e388677fe147892e2b0363519660d"} Jul 09 14:59:18.218408 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:18.218359 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" podStartSLOduration=1.218342854 podStartE2EDuration="1.218342854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:59:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:59:18.215840625 +0000 UTC m=+2308.581935011" watchObservedRunningTime="2026-07-09 14:59:18.218342854 +0000 UTC m=+2308.584437239" Jul 09 14:59:27.419981 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:27.419938 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:27.419981 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:27.419991 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:27.432919 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:27.432887 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 14:59:28.255053 ip-10-0-136-93 kubenswrapper[2582]: I0709 14:59:28.255020 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 15:00:15.079403 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:15.079373 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq"] Jul 09 15:00:20.697942 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.697899 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw"] Jul 09 15:00:20.702108 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.702087 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.704948 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.704926 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0d877f\"/\"rate-test-model-0d877f-kserve-self-signed-certs\"" Jul 09 15:00:20.710623 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.710596 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw"] Jul 09 15:00:20.716511 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.716488 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-tls-certs\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.716599 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.716527 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-tmp-dir\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.716599 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.716588 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-model-cache\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.716678 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.716630 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-dshm\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.716678 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.716659 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8tkc\" (UniqueName: \"kubernetes.io/projected/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-kube-api-access-h8tkc\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.716743 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.716688 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-home\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.817280 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.817244 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-tls-certs\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.817280 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.817284 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-tmp-dir\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.817510 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.817316 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-model-cache\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.817510 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.817346 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-dshm\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.817510 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.817365 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8tkc\" (UniqueName: \"kubernetes.io/projected/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-kube-api-access-h8tkc\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.817510 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.817392 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-home\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.817782 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.817757 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-tmp-dir\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.817782 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.817773 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-home\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.817951 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.817826 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-model-cache\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.819767 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.819745 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-dshm\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.820053 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.820035 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-tls-certs\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:20.825317 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:20.825295 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8tkc\" (UniqueName: \"kubernetes.io/projected/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-kube-api-access-h8tkc\") pod \"rate-test-model-0d877f-kserve-8475878cd7-zfxcw\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:21.012046 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:21.011960 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:21.148958 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:21.148930 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw"] Jul 09 15:00:21.151413 ip-10-0-136-93 kubenswrapper[2582]: W0709 15:00:21.151373 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43fa0c89_e8fc_4e10_aafa_3b225ffc006d.slice/crio-c3ea7d6d3d7519090063678549eeda0d44d26ee4de22076b5cab30af84dc172a WatchSource:0}: Error finding container c3ea7d6d3d7519090063678549eeda0d44d26ee4de22076b5cab30af84dc172a: Status 404 returned error can't find the container with id c3ea7d6d3d7519090063678549eeda0d44d26ee4de22076b5cab30af84dc172a Jul 09 15:00:21.484132 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:21.484098 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" event={"ID":"43fa0c89-e8fc-4e10-aafa-3b225ffc006d","Type":"ContainerStarted","Data":"7b5e3b6c72017ce2508b6d65dc5655d067636b120cffbecc326f576b2255be11"} Jul 09 15:00:21.484304 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:21.484140 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" event={"ID":"43fa0c89-e8fc-4e10-aafa-3b225ffc006d","Type":"ContainerStarted","Data":"c3ea7d6d3d7519090063678549eeda0d44d26ee4de22076b5cab30af84dc172a"} Jul 09 15:00:21.501798 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:21.501747 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" podStartSLOduration=1.5017320939999999 podStartE2EDuration="1.501732094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:00:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:00:21.500467472 +0000 UTC m=+2371.866561840" watchObservedRunningTime="2026-07-09 15:00:21.501732094 +0000 UTC m=+2371.867826492" Jul 09 15:00:30.099956 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.099911 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" podUID="b2157287-82ef-4364-87e0-e7fd8d62d1c3" containerName="main" containerID="cri-o://fc826f3b74dfb62f5766623af90eed9c79de9619261a8e2dd78903e10e7dcfeb" gracePeriod=45 Jul 09 15:00:30.358604 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.358546 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 15:00:30.401859 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.401830 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-model-cache\") pod \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " Jul 09 15:00:30.402034 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.401894 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-tmp-dir\") pod \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " Jul 09 15:00:30.402034 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.401945 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-824fg\" (UniqueName: \"kubernetes.io/projected/b2157287-82ef-4364-87e0-e7fd8d62d1c3-kube-api-access-824fg\") pod \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " Jul 09 15:00:30.402034 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.401993 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2157287-82ef-4364-87e0-e7fd8d62d1c3-tls-certs\") pod \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " Jul 09 15:00:30.402211 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.402067 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-home\") pod \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " Jul 09 15:00:30.402211 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.402089 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-dshm\") pod \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\" (UID: \"b2157287-82ef-4364-87e0-e7fd8d62d1c3\") " Jul 09 15:00:30.402211 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.402150 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-model-cache" (OuterVolumeSpecName: "model-cache") pod "b2157287-82ef-4364-87e0-e7fd8d62d1c3" (UID: "b2157287-82ef-4364-87e0-e7fd8d62d1c3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:00:30.402363 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.402315 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-model-cache\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:00:30.402417 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.402364 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-home" (OuterVolumeSpecName: "home") pod "b2157287-82ef-4364-87e0-e7fd8d62d1c3" (UID: "b2157287-82ef-4364-87e0-e7fd8d62d1c3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:00:30.402561 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.402532 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b2157287-82ef-4364-87e0-e7fd8d62d1c3" (UID: "b2157287-82ef-4364-87e0-e7fd8d62d1c3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:00:30.404530 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.404505 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-dshm" (OuterVolumeSpecName: "dshm") pod "b2157287-82ef-4364-87e0-e7fd8d62d1c3" (UID: "b2157287-82ef-4364-87e0-e7fd8d62d1c3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:00:30.404618 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.404528 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2157287-82ef-4364-87e0-e7fd8d62d1c3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b2157287-82ef-4364-87e0-e7fd8d62d1c3" (UID: "b2157287-82ef-4364-87e0-e7fd8d62d1c3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:00:30.404662 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.404623 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2157287-82ef-4364-87e0-e7fd8d62d1c3-kube-api-access-824fg" (OuterVolumeSpecName: "kube-api-access-824fg") pod "b2157287-82ef-4364-87e0-e7fd8d62d1c3" (UID: "b2157287-82ef-4364-87e0-e7fd8d62d1c3"). InnerVolumeSpecName "kube-api-access-824fg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:00:30.503156 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.503114 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-home\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:00:30.503156 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.503148 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-dshm\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:00:30.503156 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.503157 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2157287-82ef-4364-87e0-e7fd8d62d1c3-tmp-dir\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:00:30.503388 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.503167 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-824fg\" (UniqueName: \"kubernetes.io/projected/b2157287-82ef-4364-87e0-e7fd8d62d1c3-kube-api-access-824fg\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:00:30.503388 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.503178 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2157287-82ef-4364-87e0-e7fd8d62d1c3-tls-certs\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:00:30.527048 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.527018 2582 generic.go:356] "Generic (PLEG): container finished" podID="b2157287-82ef-4364-87e0-e7fd8d62d1c3" containerID="fc826f3b74dfb62f5766623af90eed9c79de9619261a8e2dd78903e10e7dcfeb" exitCode=0 Jul 09 15:00:30.527153 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.527069 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" Jul 09 15:00:30.527153 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.527100 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" event={"ID":"b2157287-82ef-4364-87e0-e7fd8d62d1c3","Type":"ContainerDied","Data":"fc826f3b74dfb62f5766623af90eed9c79de9619261a8e2dd78903e10e7dcfeb"} Jul 09 15:00:30.527153 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.527146 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq" event={"ID":"b2157287-82ef-4364-87e0-e7fd8d62d1c3","Type":"ContainerDied","Data":"05d3165809e25917d7ad5006fe339513056e388677fe147892e2b0363519660d"} Jul 09 15:00:30.527250 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.527164 2582 scope.go:117] "RemoveContainer" containerID="fc826f3b74dfb62f5766623af90eed9c79de9619261a8e2dd78903e10e7dcfeb" Jul 09 15:00:30.536338 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.536322 2582 scope.go:117] "RemoveContainer" containerID="fc826f3b74dfb62f5766623af90eed9c79de9619261a8e2dd78903e10e7dcfeb" Jul 09 15:00:30.536654 ip-10-0-136-93 kubenswrapper[2582]: E0709 15:00:30.536629 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc826f3b74dfb62f5766623af90eed9c79de9619261a8e2dd78903e10e7dcfeb\": container with ID starting with fc826f3b74dfb62f5766623af90eed9c79de9619261a8e2dd78903e10e7dcfeb not found: ID does not exist" containerID="fc826f3b74dfb62f5766623af90eed9c79de9619261a8e2dd78903e10e7dcfeb" Jul 09 15:00:30.536750 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.536661 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc826f3b74dfb62f5766623af90eed9c79de9619261a8e2dd78903e10e7dcfeb"} err="failed to get container status \"fc826f3b74dfb62f5766623af90eed9c79de9619261a8e2dd78903e10e7dcfeb\": rpc error: code = NotFound desc = could not find container \"fc826f3b74dfb62f5766623af90eed9c79de9619261a8e2dd78903e10e7dcfeb\": container with ID starting with fc826f3b74dfb62f5766623af90eed9c79de9619261a8e2dd78903e10e7dcfeb not found: ID does not exist" Jul 09 15:00:30.548823 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.548800 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq"] Jul 09 15:00:30.554505 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:30.554485 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0d877f/sub-test-model-0d877f-kserve-5946fb6555-g65qq"] Jul 09 15:00:31.012230 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:31.012188 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:31.012430 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:31.012244 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:31.024994 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:31.024963 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:31.546771 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:31.546741 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:00:32.038936 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:00:32.038895 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2157287-82ef-4364-87e0-e7fd8d62d1c3" path="/var/lib/kubelet/pods/b2157287-82ef-4364-87e0-e7fd8d62d1c3/volumes" Jul 09 15:01:41.677526 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:41.677495 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw"] Jul 09 15:01:43.696547 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:43.696512 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7425-139773529187888"] Jul 09 15:01:43.696944 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:43.696890 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2157287-82ef-4364-87e0-e7fd8d62d1c3" containerName="main" Jul 09 15:01:43.696944 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:43.696900 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2157287-82ef-4364-87e0-e7fd8d62d1c3" containerName="main" Jul 09 15:01:43.697033 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:43.696964 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2157287-82ef-4364-87e0-e7fd8d62d1c3" containerName="main" Jul 09 15:01:43.699848 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:43.699832 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773529187888" Jul 09 15:01:43.702525 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:43.702507 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-dwnwr\"" Jul 09 15:01:43.705732 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:43.705708 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7425-139773529187888"] Jul 09 15:01:43.835616 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:43.835581 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpdf2\" (UniqueName: \"kubernetes.io/projected/27cc61b8-8173-4328-a94d-e8b85c56667e-kube-api-access-rpdf2\") pod \"test-curl-7425-139773529187888\" (UID: \"27cc61b8-8173-4328-a94d-e8b85c56667e\") " pod="openshift-ingress/test-curl-7425-139773529187888" Jul 09 15:01:43.936473 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:43.936426 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpdf2\" (UniqueName: \"kubernetes.io/projected/27cc61b8-8173-4328-a94d-e8b85c56667e-kube-api-access-rpdf2\") pod \"test-curl-7425-139773529187888\" (UID: \"27cc61b8-8173-4328-a94d-e8b85c56667e\") " pod="openshift-ingress/test-curl-7425-139773529187888" Jul 09 15:01:43.945437 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:43.945409 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpdf2\" (UniqueName: \"kubernetes.io/projected/27cc61b8-8173-4328-a94d-e8b85c56667e-kube-api-access-rpdf2\") pod \"test-curl-7425-139773529187888\" (UID: \"27cc61b8-8173-4328-a94d-e8b85c56667e\") " pod="openshift-ingress/test-curl-7425-139773529187888" Jul 09 15:01:44.010431 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:44.010361 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773529187888" Jul 09 15:01:44.145324 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:44.145301 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7425-139773529187888"] Jul 09 15:01:44.147844 ip-10-0-136-93 kubenswrapper[2582]: W0709 15:01:44.147814 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27cc61b8_8173_4328_a94d_e8b85c56667e.slice/crio-0eb8e69f503bcf3cc6a46c6dbf6009fa1ff0e7d0605cff54d7fdc447e7577b6e WatchSource:0}: Error finding container 0eb8e69f503bcf3cc6a46c6dbf6009fa1ff0e7d0605cff54d7fdc447e7577b6e: Status 404 returned error can't find the container with id 0eb8e69f503bcf3cc6a46c6dbf6009fa1ff0e7d0605cff54d7fdc447e7577b6e Jul 09 15:01:44.149709 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:44.149685 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 15:01:44.870158 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:44.870123 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7425-139773529187888" event={"ID":"27cc61b8-8173-4328-a94d-e8b85c56667e","Type":"ContainerStarted","Data":"0eb8e69f503bcf3cc6a46c6dbf6009fa1ff0e7d0605cff54d7fdc447e7577b6e"} Jul 09 15:01:45.876389 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:45.876354 2582 generic.go:356] "Generic (PLEG): container finished" podID="27cc61b8-8173-4328-a94d-e8b85c56667e" containerID="17b247f0293ac100b26e922d4746f9a201a506348e83c66863ed4565f64fdfb6" exitCode=0 Jul 09 15:01:45.876840 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:45.876406 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7425-139773529187888" event={"ID":"27cc61b8-8173-4328-a94d-e8b85c56667e","Type":"ContainerDied","Data":"17b247f0293ac100b26e922d4746f9a201a506348e83c66863ed4565f64fdfb6"} Jul 09 15:01:47.021773 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.021750 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773529187888" Jul 09 15:01:47.062490 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.062464 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7425-139773529187888_27cc61b8-8173-4328-a94d-e8b85c56667e/test-curl-7425-139773529187888/0.log" Jul 09 15:01:47.088088 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.088060 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7425-139773529187888"] Jul 09 15:01:47.092352 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.092333 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7425-139773529187888"] Jul 09 15:01:47.166440 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.166371 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpdf2\" (UniqueName: \"kubernetes.io/projected/27cc61b8-8173-4328-a94d-e8b85c56667e-kube-api-access-rpdf2\") pod \"27cc61b8-8173-4328-a94d-e8b85c56667e\" (UID: \"27cc61b8-8173-4328-a94d-e8b85c56667e\") " Jul 09 15:01:47.168713 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.168680 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27cc61b8-8173-4328-a94d-e8b85c56667e-kube-api-access-rpdf2" (OuterVolumeSpecName: "kube-api-access-rpdf2") pod "27cc61b8-8173-4328-a94d-e8b85c56667e" (UID: "27cc61b8-8173-4328-a94d-e8b85c56667e"). InnerVolumeSpecName "kube-api-access-rpdf2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:01:47.215907 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.215860 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7425-139773529187120"] Jul 09 15:01:47.216348 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.216285 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27cc61b8-8173-4328-a94d-e8b85c56667e" containerName="test-curl-7425-139773529187888" Jul 09 15:01:47.216348 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.216307 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="27cc61b8-8173-4328-a94d-e8b85c56667e" containerName="test-curl-7425-139773529187888" Jul 09 15:01:47.216557 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.216395 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="27cc61b8-8173-4328-a94d-e8b85c56667e" containerName="test-curl-7425-139773529187888" Jul 09 15:01:47.219485 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.219465 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773529187120" Jul 09 15:01:47.226182 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.226158 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7425-139773529187120"] Jul 09 15:01:47.267956 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.267923 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpdf2\" (UniqueName: \"kubernetes.io/projected/27cc61b8-8173-4328-a94d-e8b85c56667e-kube-api-access-rpdf2\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:01:47.368787 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.368756 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpgbp\" (UniqueName: \"kubernetes.io/projected/8fe43547-95e2-46c9-9c2a-bc4ddcc38de8-kube-api-access-cpgbp\") pod \"test-curl-7425-139773529187120\" (UID: \"8fe43547-95e2-46c9-9c2a-bc4ddcc38de8\") " pod="openshift-ingress/test-curl-7425-139773529187120" Jul 09 15:01:47.469291 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.469205 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpgbp\" (UniqueName: \"kubernetes.io/projected/8fe43547-95e2-46c9-9c2a-bc4ddcc38de8-kube-api-access-cpgbp\") pod \"test-curl-7425-139773529187120\" (UID: \"8fe43547-95e2-46c9-9c2a-bc4ddcc38de8\") " pod="openshift-ingress/test-curl-7425-139773529187120" Jul 09 15:01:47.477862 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.477829 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpgbp\" (UniqueName: \"kubernetes.io/projected/8fe43547-95e2-46c9-9c2a-bc4ddcc38de8-kube-api-access-cpgbp\") pod \"test-curl-7425-139773529187120\" (UID: \"8fe43547-95e2-46c9-9c2a-bc4ddcc38de8\") " pod="openshift-ingress/test-curl-7425-139773529187120" Jul 09 15:01:47.529724 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.529695 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773529187120" Jul 09 15:01:47.662631 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.662593 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7425-139773529187120"] Jul 09 15:01:47.666524 ip-10-0-136-93 kubenswrapper[2582]: W0709 15:01:47.666497 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8fe43547_95e2_46c9_9c2a_bc4ddcc38de8.slice/crio-e0b2a85deacdf7eaa02267cf750a3e7beb708dd73295c5fedacf094ad655b36d WatchSource:0}: Error finding container e0b2a85deacdf7eaa02267cf750a3e7beb708dd73295c5fedacf094ad655b36d: Status 404 returned error can't find the container with id e0b2a85deacdf7eaa02267cf750a3e7beb708dd73295c5fedacf094ad655b36d Jul 09 15:01:47.887202 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.887165 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7425-139773529187120" event={"ID":"8fe43547-95e2-46c9-9c2a-bc4ddcc38de8","Type":"ContainerStarted","Data":"e0b2a85deacdf7eaa02267cf750a3e7beb708dd73295c5fedacf094ad655b36d"} Jul 09 15:01:47.888312 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.888294 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0eb8e69f503bcf3cc6a46c6dbf6009fa1ff0e7d0605cff54d7fdc447e7577b6e" Jul 09 15:01:47.888418 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:47.888313 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773529187888" Jul 09 15:01:47.898336 ip-10-0-136-93 kubenswrapper[2582]: E0709 15:01:47.898304 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7425-139773529187888\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="27cc61b8-8173-4328-a94d-e8b85c56667e" pod="openshift-ingress/test-curl-7425-139773529187888" Jul 09 15:01:48.037430 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:48.037406 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27cc61b8-8173-4328-a94d-e8b85c56667e" path="/var/lib/kubelet/pods/27cc61b8-8173-4328-a94d-e8b85c56667e/volumes" Jul 09 15:01:48.895698 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:48.895660 2582 generic.go:356] "Generic (PLEG): container finished" podID="8fe43547-95e2-46c9-9c2a-bc4ddcc38de8" containerID="c70836f63c4635b8b2a3e3dd653dc7186fedfc5a3feee0e7a39e89d1f6fdc83b" exitCode=0 Jul 09 15:01:48.895926 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:48.895717 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7425-139773529187120" event={"ID":"8fe43547-95e2-46c9-9c2a-bc4ddcc38de8","Type":"ContainerDied","Data":"c70836f63c4635b8b2a3e3dd653dc7186fedfc5a3feee0e7a39e89d1f6fdc83b"} Jul 09 15:01:50.047668 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.047643 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773529187120" Jul 09 15:01:50.068031 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.068010 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7425-139773529187120_8fe43547-95e2-46c9-9c2a-bc4ddcc38de8/test-curl-7425-139773529187120/0.log" Jul 09 15:01:50.094619 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.094596 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7425-139773529187120"] Jul 09 15:01:50.098375 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.098354 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7425-139773529187120"] Jul 09 15:01:50.193245 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.193162 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpgbp\" (UniqueName: \"kubernetes.io/projected/8fe43547-95e2-46c9-9c2a-bc4ddcc38de8-kube-api-access-cpgbp\") pod \"8fe43547-95e2-46c9-9c2a-bc4ddcc38de8\" (UID: \"8fe43547-95e2-46c9-9c2a-bc4ddcc38de8\") " Jul 09 15:01:50.195430 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.195399 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fe43547-95e2-46c9-9c2a-bc4ddcc38de8-kube-api-access-cpgbp" (OuterVolumeSpecName: "kube-api-access-cpgbp") pod "8fe43547-95e2-46c9-9c2a-bc4ddcc38de8" (UID: "8fe43547-95e2-46c9-9c2a-bc4ddcc38de8"). InnerVolumeSpecName "kube-api-access-cpgbp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:01:50.264033 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.264000 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7425-139773530486080"] Jul 09 15:01:50.264365 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.264353 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fe43547-95e2-46c9-9c2a-bc4ddcc38de8" containerName="test-curl-7425-139773529187120" Jul 09 15:01:50.264405 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.264367 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fe43547-95e2-46c9-9c2a-bc4ddcc38de8" containerName="test-curl-7425-139773529187120" Jul 09 15:01:50.264439 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.264434 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fe43547-95e2-46c9-9c2a-bc4ddcc38de8" containerName="test-curl-7425-139773529187120" Jul 09 15:01:50.267390 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.267373 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:50.273273 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.273248 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7425-139773530486080"] Jul 09 15:01:50.293959 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.293926 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpgbp\" (UniqueName: \"kubernetes.io/projected/8fe43547-95e2-46c9-9c2a-bc4ddcc38de8-kube-api-access-cpgbp\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:01:50.394520 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.394494 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q8ll\" (UniqueName: \"kubernetes.io/projected/51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4-kube-api-access-9q8ll\") pod \"test-curl-7425-139773530486080\" (UID: \"51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4\") " pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:50.495434 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.495365 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q8ll\" (UniqueName: \"kubernetes.io/projected/51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4-kube-api-access-9q8ll\") pod \"test-curl-7425-139773530486080\" (UID: \"51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4\") " pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:50.503628 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.503600 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q8ll\" (UniqueName: \"kubernetes.io/projected/51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4-kube-api-access-9q8ll\") pod \"test-curl-7425-139773530486080\" (UID: \"51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4\") " pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:50.577658 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.577630 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:50.711203 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.711178 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7425-139773530486080"] Jul 09 15:01:50.713797 ip-10-0-136-93 kubenswrapper[2582]: W0709 15:01:50.713761 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51b6aa1f_5e68_42dd_96cd_eaf3b4c558d4.slice/crio-ce46b587a02a101be1484ae457101b385c9ee774c0f1ec791c0359f58c484e38 WatchSource:0}: Error finding container ce46b587a02a101be1484ae457101b385c9ee774c0f1ec791c0359f58c484e38: Status 404 returned error can't find the container with id ce46b587a02a101be1484ae457101b385c9ee774c0f1ec791c0359f58c484e38 Jul 09 15:01:50.906686 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.906658 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0b2a85deacdf7eaa02267cf750a3e7beb708dd73295c5fedacf094ad655b36d" Jul 09 15:01:50.906686 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.906661 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773529187120" Jul 09 15:01:50.907866 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:50.907840 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7425-139773530486080" event={"ID":"51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4","Type":"ContainerStarted","Data":"ce46b587a02a101be1484ae457101b385c9ee774c0f1ec791c0359f58c484e38"} Jul 09 15:01:50.917041 ip-10-0-136-93 kubenswrapper[2582]: E0709 15:01:50.917016 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7425-139773529187120\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="8fe43547-95e2-46c9-9c2a-bc4ddcc38de8" pod="openshift-ingress/test-curl-7425-139773529187120" Jul 09 15:01:51.914643 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:51.914605 2582 generic.go:356] "Generic (PLEG): container finished" podID="51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4" containerID="5db74b013df5ac7c3ad06c852fbc923ae7d222c34fe2bd75da7cc3a4c0661555" exitCode=0 Jul 09 15:01:51.915086 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:51.914653 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7425-139773530486080" event={"ID":"51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4","Type":"ContainerDied","Data":"5db74b013df5ac7c3ad06c852fbc923ae7d222c34fe2bd75da7cc3a4c0661555"} Jul 09 15:01:51.917194 ip-10-0-136-93 kubenswrapper[2582]: E0709 15:01:51.917167 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7425-139773529187120\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="8fe43547-95e2-46c9-9c2a-bc4ddcc38de8" pod="openshift-ingress/test-curl-7425-139773529187120" Jul 09 15:01:52.037867 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:52.037836 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fe43547-95e2-46c9-9c2a-bc4ddcc38de8" path="/var/lib/kubelet/pods/8fe43547-95e2-46c9-9c2a-bc4ddcc38de8/volumes" Jul 09 15:01:53.069745 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.069716 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:53.092204 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.092175 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7425-139773530486080_51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4/test-curl-7425-139773530486080/0.log" Jul 09 15:01:53.119181 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.119157 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7425-139773530486080"] Jul 09 15:01:53.123229 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.123209 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7425-139773530486080"] Jul 09 15:01:53.219731 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.219657 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q8ll\" (UniqueName: \"kubernetes.io/projected/51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4-kube-api-access-9q8ll\") pod \"51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4\" (UID: \"51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4\") " Jul 09 15:01:53.221815 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.221788 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4-kube-api-access-9q8ll" (OuterVolumeSpecName: "kube-api-access-9q8ll") pod "51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4" (UID: "51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4"). InnerVolumeSpecName "kube-api-access-9q8ll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:01:53.250739 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.250714 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7425-139773530486080"] Jul 09 15:01:53.251076 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.251064 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4" containerName="test-curl-7425-139773530486080" Jul 09 15:01:53.251122 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.251078 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4" containerName="test-curl-7425-139773530486080" Jul 09 15:01:53.251156 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.251147 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4" containerName="test-curl-7425-139773530486080" Jul 09 15:01:53.254084 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.254065 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:53.261532 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.261496 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7425-139773530486080"] Jul 09 15:01:53.321145 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.321115 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q8ll\" (UniqueName: \"kubernetes.io/projected/51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4-kube-api-access-9q8ll\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:01:53.421923 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.421894 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xwsv\" (UniqueName: \"kubernetes.io/projected/5721f9e1-950b-4d02-adc0-8f3db6509290-kube-api-access-2xwsv\") pod \"test-curl-7425-139773530486080\" (UID: \"5721f9e1-950b-4d02-adc0-8f3db6509290\") " pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:53.523240 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.523156 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xwsv\" (UniqueName: \"kubernetes.io/projected/5721f9e1-950b-4d02-adc0-8f3db6509290-kube-api-access-2xwsv\") pod \"test-curl-7425-139773530486080\" (UID: \"5721f9e1-950b-4d02-adc0-8f3db6509290\") " pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:53.531078 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.531053 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xwsv\" (UniqueName: \"kubernetes.io/projected/5721f9e1-950b-4d02-adc0-8f3db6509290-kube-api-access-2xwsv\") pod \"test-curl-7425-139773530486080\" (UID: \"5721f9e1-950b-4d02-adc0-8f3db6509290\") " pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:53.566065 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.566028 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:53.697114 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.697089 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7425-139773530486080"] Jul 09 15:01:53.699164 ip-10-0-136-93 kubenswrapper[2582]: W0709 15:01:53.699139 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5721f9e1_950b_4d02_adc0_8f3db6509290.slice/crio-208d8fd873827f0a04702487bdd8964a7510b6efbfac78c5040aa9dd94339a41 WatchSource:0}: Error finding container 208d8fd873827f0a04702487bdd8964a7510b6efbfac78c5040aa9dd94339a41: Status 404 returned error can't find the container with id 208d8fd873827f0a04702487bdd8964a7510b6efbfac78c5040aa9dd94339a41 Jul 09 15:01:53.927214 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.927181 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7425-139773530486080" event={"ID":"5721f9e1-950b-4d02-adc0-8f3db6509290","Type":"ContainerStarted","Data":"208d8fd873827f0a04702487bdd8964a7510b6efbfac78c5040aa9dd94339a41"} Jul 09 15:01:53.928422 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.928399 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce46b587a02a101be1484ae457101b385c9ee774c0f1ec791c0359f58c484e38" Jul 09 15:01:53.928422 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.928418 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:53.939189 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:53.939157 2582 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7425-139773530486080" oldPodUID="51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4" podUID="5721f9e1-950b-4d02-adc0-8f3db6509290" Jul 09 15:01:54.038814 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:54.038778 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4" path="/var/lib/kubelet/pods/51b6aa1f-5e68-42dd-96cd-eaf3b4c558d4/volumes" Jul 09 15:01:54.934724 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:54.934690 2582 generic.go:356] "Generic (PLEG): container finished" podID="5721f9e1-950b-4d02-adc0-8f3db6509290" containerID="e7d5e5a44f4bde670aa5dac9463e7179329771a3923824b5a64c1377412611f0" exitCode=0 Jul 09 15:01:54.935126 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:54.934779 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7425-139773530486080" event={"ID":"5721f9e1-950b-4d02-adc0-8f3db6509290","Type":"ContainerDied","Data":"e7d5e5a44f4bde670aa5dac9463e7179329771a3923824b5a64c1377412611f0"} Jul 09 15:01:56.080417 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.080394 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:56.100314 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.100292 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7425-139773530486080_5721f9e1-950b-4d02-adc0-8f3db6509290/test-curl-7425-139773530486080/0.log" Jul 09 15:01:56.124655 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.124623 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7425-139773530486080"] Jul 09 15:01:56.127415 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.127390 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7425-139773530486080"] Jul 09 15:01:56.247470 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.247386 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7425-139773530487952"] Jul 09 15:01:56.247752 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.247739 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5721f9e1-950b-4d02-adc0-8f3db6509290" containerName="test-curl-7425-139773530486080" Jul 09 15:01:56.247794 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.247753 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5721f9e1-950b-4d02-adc0-8f3db6509290" containerName="test-curl-7425-139773530486080" Jul 09 15:01:56.247824 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.247816 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5721f9e1-950b-4d02-adc0-8f3db6509290" containerName="test-curl-7425-139773530486080" Jul 09 15:01:56.248724 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.248707 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xwsv\" (UniqueName: \"kubernetes.io/projected/5721f9e1-950b-4d02-adc0-8f3db6509290-kube-api-access-2xwsv\") pod \"5721f9e1-950b-4d02-adc0-8f3db6509290\" (UID: \"5721f9e1-950b-4d02-adc0-8f3db6509290\") " Jul 09 15:01:56.250775 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.250755 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773530487952" Jul 09 15:01:56.251111 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.251090 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5721f9e1-950b-4d02-adc0-8f3db6509290-kube-api-access-2xwsv" (OuterVolumeSpecName: "kube-api-access-2xwsv") pod "5721f9e1-950b-4d02-adc0-8f3db6509290" (UID: "5721f9e1-950b-4d02-adc0-8f3db6509290"). InnerVolumeSpecName "kube-api-access-2xwsv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:01:56.256018 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.256000 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7425-139773530487952"] Jul 09 15:01:56.349782 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.349715 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfdbh\" (UniqueName: \"kubernetes.io/projected/2e53bdcc-0e65-4b04-b72b-32dc3f95eb45-kube-api-access-wfdbh\") pod \"test-curl-7425-139773530487952\" (UID: \"2e53bdcc-0e65-4b04-b72b-32dc3f95eb45\") " pod="openshift-ingress/test-curl-7425-139773530487952" Jul 09 15:01:56.350083 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.349999 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xwsv\" (UniqueName: \"kubernetes.io/projected/5721f9e1-950b-4d02-adc0-8f3db6509290-kube-api-access-2xwsv\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:01:56.450659 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.450624 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfdbh\" (UniqueName: \"kubernetes.io/projected/2e53bdcc-0e65-4b04-b72b-32dc3f95eb45-kube-api-access-wfdbh\") pod \"test-curl-7425-139773530487952\" (UID: \"2e53bdcc-0e65-4b04-b72b-32dc3f95eb45\") " pod="openshift-ingress/test-curl-7425-139773530487952" Jul 09 15:01:56.458783 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.458756 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfdbh\" (UniqueName: \"kubernetes.io/projected/2e53bdcc-0e65-4b04-b72b-32dc3f95eb45-kube-api-access-wfdbh\") pod \"test-curl-7425-139773530487952\" (UID: \"2e53bdcc-0e65-4b04-b72b-32dc3f95eb45\") " pod="openshift-ingress/test-curl-7425-139773530487952" Jul 09 15:01:56.567858 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.567823 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773530487952" Jul 09 15:01:56.698108 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.698054 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" podUID="43fa0c89-e8fc-4e10-aafa-3b225ffc006d" containerName="main" containerID="cri-o://7b5e3b6c72017ce2508b6d65dc5655d067636b120cffbecc326f576b2255be11" gracePeriod=45 Jul 09 15:01:56.700549 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.700471 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7425-139773530487952"] Jul 09 15:01:56.703685 ip-10-0-136-93 kubenswrapper[2582]: W0709 15:01:56.703660 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e53bdcc_0e65_4b04_b72b_32dc3f95eb45.slice/crio-8ec0fb8ea27dbf9e1305cf7f972bfb060c77140d1e1c517cc7051cef59f7c001 WatchSource:0}: Error finding container 8ec0fb8ea27dbf9e1305cf7f972bfb060c77140d1e1c517cc7051cef59f7c001: Status 404 returned error can't find the container with id 8ec0fb8ea27dbf9e1305cf7f972bfb060c77140d1e1c517cc7051cef59f7c001 Jul 09 15:01:56.944412 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.944385 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:01:56.946430 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.946399 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7425-139773530487952" event={"ID":"2e53bdcc-0e65-4b04-b72b-32dc3f95eb45","Type":"ContainerStarted","Data":"8ec0fb8ea27dbf9e1305cf7f972bfb060c77140d1e1c517cc7051cef59f7c001"} Jul 09 15:01:56.948292 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.948271 2582 generic.go:356] "Generic (PLEG): container finished" podID="43fa0c89-e8fc-4e10-aafa-3b225ffc006d" containerID="7b5e3b6c72017ce2508b6d65dc5655d067636b120cffbecc326f576b2255be11" exitCode=0 Jul 09 15:01:56.948394 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.948312 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" Jul 09 15:01:56.948394 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.948349 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" event={"ID":"43fa0c89-e8fc-4e10-aafa-3b225ffc006d","Type":"ContainerDied","Data":"7b5e3b6c72017ce2508b6d65dc5655d067636b120cffbecc326f576b2255be11"} Jul 09 15:01:56.948394 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.948379 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw" event={"ID":"43fa0c89-e8fc-4e10-aafa-3b225ffc006d","Type":"ContainerDied","Data":"c3ea7d6d3d7519090063678549eeda0d44d26ee4de22076b5cab30af84dc172a"} Jul 09 15:01:56.948525 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.948396 2582 scope.go:117] "RemoveContainer" containerID="7b5e3b6c72017ce2508b6d65dc5655d067636b120cffbecc326f576b2255be11" Jul 09 15:01:56.949689 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.949665 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:56.949689 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.949678 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="208d8fd873827f0a04702487bdd8964a7510b6efbfac78c5040aa9dd94339a41" Jul 09 15:01:56.956453 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.956437 2582 scope.go:117] "RemoveContainer" containerID="7b5e3b6c72017ce2508b6d65dc5655d067636b120cffbecc326f576b2255be11" Jul 09 15:01:56.956763 ip-10-0-136-93 kubenswrapper[2582]: E0709 15:01:56.956736 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b5e3b6c72017ce2508b6d65dc5655d067636b120cffbecc326f576b2255be11\": container with ID starting with 7b5e3b6c72017ce2508b6d65dc5655d067636b120cffbecc326f576b2255be11 not found: ID does not exist" containerID="7b5e3b6c72017ce2508b6d65dc5655d067636b120cffbecc326f576b2255be11" Jul 09 15:01:56.956828 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:56.956776 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b5e3b6c72017ce2508b6d65dc5655d067636b120cffbecc326f576b2255be11"} err="failed to get container status \"7b5e3b6c72017ce2508b6d65dc5655d067636b120cffbecc326f576b2255be11\": rpc error: code = NotFound desc = could not find container \"7b5e3b6c72017ce2508b6d65dc5655d067636b120cffbecc326f576b2255be11\": container with ID starting with 7b5e3b6c72017ce2508b6d65dc5655d067636b120cffbecc326f576b2255be11 not found: ID does not exist" Jul 09 15:01:56.966555 ip-10-0-136-93 kubenswrapper[2582]: E0709 15:01:56.966525 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7425-139773530486080\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="5721f9e1-950b-4d02-adc0-8f3db6509290" pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:57.055791 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.055757 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-home\") pod \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " Jul 09 15:01:57.055938 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.055833 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-tmp-dir\") pod \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " Jul 09 15:01:57.055938 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.055860 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-tls-certs\") pod \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " Jul 09 15:01:57.056062 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.055949 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-model-cache\") pod \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " Jul 09 15:01:57.056062 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.055982 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8tkc\" (UniqueName: \"kubernetes.io/projected/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-kube-api-access-h8tkc\") pod \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " Jul 09 15:01:57.056062 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.056031 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-dshm\") pod \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\" (UID: \"43fa0c89-e8fc-4e10-aafa-3b225ffc006d\") " Jul 09 15:01:57.056220 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.056023 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-home" (OuterVolumeSpecName: "home") pod "43fa0c89-e8fc-4e10-aafa-3b225ffc006d" (UID: "43fa0c89-e8fc-4e10-aafa-3b225ffc006d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:01:57.056220 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.056116 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "43fa0c89-e8fc-4e10-aafa-3b225ffc006d" (UID: "43fa0c89-e8fc-4e10-aafa-3b225ffc006d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:01:57.056420 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.056258 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-model-cache" (OuterVolumeSpecName: "model-cache") pod "43fa0c89-e8fc-4e10-aafa-3b225ffc006d" (UID: "43fa0c89-e8fc-4e10-aafa-3b225ffc006d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:01:57.056505 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.056483 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-home\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:01:57.056556 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.056511 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-tmp-dir\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:01:57.056556 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.056525 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-model-cache\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:01:57.058335 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.058305 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-dshm" (OuterVolumeSpecName: "dshm") pod "43fa0c89-e8fc-4e10-aafa-3b225ffc006d" (UID: "43fa0c89-e8fc-4e10-aafa-3b225ffc006d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 15:01:57.058413 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.058359 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "43fa0c89-e8fc-4e10-aafa-3b225ffc006d" (UID: "43fa0c89-e8fc-4e10-aafa-3b225ffc006d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 15:01:57.058413 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.058361 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-kube-api-access-h8tkc" (OuterVolumeSpecName: "kube-api-access-h8tkc") pod "43fa0c89-e8fc-4e10-aafa-3b225ffc006d" (UID: "43fa0c89-e8fc-4e10-aafa-3b225ffc006d"). InnerVolumeSpecName "kube-api-access-h8tkc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:01:57.157120 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.157097 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8tkc\" (UniqueName: \"kubernetes.io/projected/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-kube-api-access-h8tkc\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:01:57.157120 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.157121 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-dshm\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:01:57.157431 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.157132 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43fa0c89-e8fc-4e10-aafa-3b225ffc006d-tls-certs\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:01:57.270477 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.270448 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw"] Jul 09 15:01:57.273887 ip-10-0-136-93 kubenswrapper[2582]: E0709 15:01:57.273837 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7425-139773530486080\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="5721f9e1-950b-4d02-adc0-8f3db6509290" pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:57.274390 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.274372 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0d877f/rate-test-model-0d877f-kserve-8475878cd7-zfxcw"] Jul 09 15:01:57.957182 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.957150 2582 generic.go:356] "Generic (PLEG): container finished" podID="2e53bdcc-0e65-4b04-b72b-32dc3f95eb45" containerID="f16a4868f4cf6b8e9097accd85b19e0ff83eef145f702cffd6b743f37e97f28e" exitCode=0 Jul 09 15:01:57.957346 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:57.957204 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7425-139773530487952" event={"ID":"2e53bdcc-0e65-4b04-b72b-32dc3f95eb45","Type":"ContainerDied","Data":"f16a4868f4cf6b8e9097accd85b19e0ff83eef145f702cffd6b743f37e97f28e"} Jul 09 15:01:57.959354 ip-10-0-136-93 kubenswrapper[2582]: E0709 15:01:57.959328 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7425-139773530486080\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="5721f9e1-950b-4d02-adc0-8f3db6509290" pod="openshift-ingress/test-curl-7425-139773530486080" Jul 09 15:01:58.037042 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:58.037009 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43fa0c89-e8fc-4e10-aafa-3b225ffc006d" path="/var/lib/kubelet/pods/43fa0c89-e8fc-4e10-aafa-3b225ffc006d/volumes" Jul 09 15:01:58.037375 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:58.037363 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5721f9e1-950b-4d02-adc0-8f3db6509290" path="/var/lib/kubelet/pods/5721f9e1-950b-4d02-adc0-8f3db6509290/volumes" Jul 09 15:01:59.116986 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:59.116965 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773530487952" Jul 09 15:01:59.137430 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:59.137403 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7425-139773530487952_2e53bdcc-0e65-4b04-b72b-32dc3f95eb45/test-curl-7425-139773530487952/0.log" Jul 09 15:01:59.161730 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:59.161706 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7425-139773530487952"] Jul 09 15:01:59.165617 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:59.165595 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7425-139773530487952"] Jul 09 15:01:59.276283 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:59.276208 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfdbh\" (UniqueName: \"kubernetes.io/projected/2e53bdcc-0e65-4b04-b72b-32dc3f95eb45-kube-api-access-wfdbh\") pod \"2e53bdcc-0e65-4b04-b72b-32dc3f95eb45\" (UID: \"2e53bdcc-0e65-4b04-b72b-32dc3f95eb45\") " Jul 09 15:01:59.278508 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:59.278480 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e53bdcc-0e65-4b04-b72b-32dc3f95eb45-kube-api-access-wfdbh" (OuterVolumeSpecName: "kube-api-access-wfdbh") pod "2e53bdcc-0e65-4b04-b72b-32dc3f95eb45" (UID: "2e53bdcc-0e65-4b04-b72b-32dc3f95eb45"). InnerVolumeSpecName "kube-api-access-wfdbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 15:01:59.377635 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:59.377603 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfdbh\" (UniqueName: \"kubernetes.io/projected/2e53bdcc-0e65-4b04-b72b-32dc3f95eb45-kube-api-access-wfdbh\") on node \"ip-10-0-136-93.ec2.internal\" DevicePath \"\"" Jul 09 15:01:59.968191 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:59.968164 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7425-139773530487952" Jul 09 15:01:59.968191 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:01:59.968180 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ec0fb8ea27dbf9e1305cf7f972bfb060c77140d1e1c517cc7051cef59f7c001" Jul 09 15:01:59.981136 ip-10-0-136-93 kubenswrapper[2582]: E0709 15:01:59.981102 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7425-139773530487952\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="2e53bdcc-0e65-4b04-b72b-32dc3f95eb45" pod="openshift-ingress/test-curl-7425-139773530487952" Jul 09 15:02:00.036470 ip-10-0-136-93 kubenswrapper[2582]: E0709 15:02:00.036430 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7425-139773530487952\" is forbidden: User \"system:node:ip-10-0-136-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-93.ec2.internal' and this object" podUID="2e53bdcc-0e65-4b04-b72b-32dc3f95eb45" pod="openshift-ingress/test-curl-7425-139773530487952" Jul 09 15:02:00.038039 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:00.038012 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e53bdcc-0e65-4b04-b72b-32dc3f95eb45" path="/var/lib/kubelet/pods/2e53bdcc-0e65-4b04-b72b-32dc3f95eb45/volumes" Jul 09 15:02:22.560538 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:22.560505 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tpgz5_980c62dc-ce17-4ac3-b661-e9a4faefdbc2/manager/0.log" Jul 09 15:02:22.678154 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:22.678119 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-95czt_0e7819cf-6cc4-4597-8e34-b5fb8a86101a/manager/0.log" Jul 09 15:02:22.944726 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:22.944643 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4fwdr_59035872-ab43-47f9-bcfe-5e1bbb034078/server/0.log" Jul 09 15:02:23.066300 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:23.066273 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-95rmq_4536ea27-ba7e-41f2-acbd-d9dac53ce64c/manager/0.log" Jul 09 15:02:23.191696 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:23.191664 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7757d956-5t57r_3568bfcb-c3d3-46ff-bb99-7d975be2d2b7/manager/0.log" Jul 09 15:02:23.200478 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:23.200414 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7757d956-5t57r_3568bfcb-c3d3-46ff-bb99-7d975be2d2b7/copy-manifests/0.log" Jul 09 15:02:23.923683 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:23.923656 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-lfzsg_c9a5b486-912e-4fc7-b78d-1016b0040365/postgres/0.log" Jul 09 15:02:25.352830 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:25.352795 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-q5t8t_1201ce06-31e0-46ac-8a6a-f816710be1dc/manager/0.log" Jul 09 15:02:25.467106 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:25.467072 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-skzpv_2e596c25-02be-45a3-a7d2-aa76614bee03/manager/0.log" Jul 09 15:02:25.585995 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:25.585970 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-djh9k_359be0dc-ada2-4058-aa4d-bee8b34419d3/kuadrant-console-plugin/0.log" Jul 09 15:02:25.705548 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:25.705480 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pjhd7_4ca4616b-0e80-4635-a6bc-1cbd8d14602b/registry-server/0.log" Jul 09 15:02:25.830458 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:25.830420 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6v7fb_484d17fb-4c43-4cda-b8a3-51885ad03ab6/manager/0.log" Jul 09 15:02:26.073691 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:26.073660 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-czvbd_7cccb007-377a-4a3c-8339-ac57ab28c5e1/manager/0.log" Jul 09 15:02:26.670985 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:26.670954 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77975789f4-69c6k_fa562d9c-ec3c-40b5-925f-a6862eab8006/kube-auth-proxy/0.log" Jul 09 15:02:27.029149 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:27.029077 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-stddl_4cdb3207-2563-400d-9dcc-043597f8e0e0/payload-pre-processing/0.log" Jul 09 15:02:27.151578 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:27.151553 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-xhrkm_633c7032-c50e-4182-bd2d-fee0d920768e/payload-processing/0.log" Jul 09 15:02:27.724665 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:27.724637 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-6zcjg_bffaa6e8-e33e-4493-aeb2-973eb6ab5ffb/main/0.log" Jul 09 15:02:27.951560 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:27.951533 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh6gmc_0a8eab4b-0985-4366-8a9c-2d8c551fc29c/main/0.log" Jul 09 15:02:28.063794 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:28.063768 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-49kpc_6f4278e9-b4c4-4c63-9433-fb5e012e1c23/main/0.log" Jul 09 15:02:34.761899 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:34.761839 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8mtbj_01f02794-646c-48f8-8fb1-85cd0a5f9882/global-pull-secret-syncer/0.log" Jul 09 15:02:34.894527 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:34.894495 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jc75x_d27fcdf2-e427-43f3-a959-a083bfc76009/konnectivity-agent/0.log" Jul 09 15:02:34.969476 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:34.969451 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-93.ec2.internal_0b2614806c45fdfd93e51f60f22bdd4c/haproxy/0.log" Jul 09 15:02:40.573675 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:40.573601 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-q5t8t_1201ce06-31e0-46ac-8a6a-f816710be1dc/manager/0.log" Jul 09 15:02:40.603346 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:40.603310 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-skzpv_2e596c25-02be-45a3-a7d2-aa76614bee03/manager/0.log" Jul 09 15:02:40.641070 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:40.641048 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-djh9k_359be0dc-ada2-4058-aa4d-bee8b34419d3/kuadrant-console-plugin/0.log" Jul 09 15:02:40.699408 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:40.699382 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pjhd7_4ca4616b-0e80-4635-a6bc-1cbd8d14602b/registry-server/0.log" Jul 09 15:02:40.796078 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:40.796052 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6v7fb_484d17fb-4c43-4cda-b8a3-51885ad03ab6/manager/0.log" Jul 09 15:02:40.899221 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:40.899144 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-czvbd_7cccb007-377a-4a3c-8339-ac57ab28c5e1/manager/0.log" Jul 09 15:02:43.032718 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:43.032686 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tjjqc_f7ae80ed-5d58-4389-85a9-0fb2171296c0/node-exporter/0.log" Jul 09 15:02:43.076903 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:43.076861 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tjjqc_f7ae80ed-5d58-4389-85a9-0fb2171296c0/kube-rbac-proxy/0.log" Jul 09 15:02:43.114591 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:43.114565 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tjjqc_f7ae80ed-5d58-4389-85a9-0fb2171296c0/init-textfile/0.log" Jul 09 15:02:46.879653 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.879623 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw"] Jul 09 15:02:46.880126 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.880025 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e53bdcc-0e65-4b04-b72b-32dc3f95eb45" containerName="test-curl-7425-139773530487952" Jul 09 15:02:46.880126 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.880038 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e53bdcc-0e65-4b04-b72b-32dc3f95eb45" containerName="test-curl-7425-139773530487952" Jul 09 15:02:46.880126 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.880047 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43fa0c89-e8fc-4e10-aafa-3b225ffc006d" containerName="main" Jul 09 15:02:46.880126 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.880053 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="43fa0c89-e8fc-4e10-aafa-3b225ffc006d" containerName="main" Jul 09 15:02:46.880126 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.880110 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e53bdcc-0e65-4b04-b72b-32dc3f95eb45" containerName="test-curl-7425-139773530487952" Jul 09 15:02:46.880126 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.880120 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="43fa0c89-e8fc-4e10-aafa-3b225ffc006d" containerName="main" Jul 09 15:02:46.883147 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.883131 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:46.885971 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.885932 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ltzpn\"/\"openshift-service-ca.crt\"" Jul 09 15:02:46.887173 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.887151 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ltzpn\"/\"kube-root-ca.crt\"" Jul 09 15:02:46.887300 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.887236 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ltzpn\"/\"default-dockercfg-pdr4l\"" Jul 09 15:02:46.892979 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.892956 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw"] Jul 09 15:02:46.986588 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.986550 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74sd5\" (UniqueName: \"kubernetes.io/projected/37d1e678-8d49-4a40-9553-d3ea84111540-kube-api-access-74sd5\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:46.986588 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.986590 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37d1e678-8d49-4a40-9553-d3ea84111540-lib-modules\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:46.986822 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.986625 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/37d1e678-8d49-4a40-9553-d3ea84111540-podres\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:46.986822 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.986701 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d1e678-8d49-4a40-9553-d3ea84111540-sys\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:46.986822 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:46.986753 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/37d1e678-8d49-4a40-9553-d3ea84111540-proc\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:47.087763 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:47.087716 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/37d1e678-8d49-4a40-9553-d3ea84111540-podres\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:47.087952 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:47.087783 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d1e678-8d49-4a40-9553-d3ea84111540-sys\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:47.087952 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:47.087821 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/37d1e678-8d49-4a40-9553-d3ea84111540-proc\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:47.087952 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:47.087847 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74sd5\" (UniqueName: \"kubernetes.io/projected/37d1e678-8d49-4a40-9553-d3ea84111540-kube-api-access-74sd5\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:47.087952 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:47.087867 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37d1e678-8d49-4a40-9553-d3ea84111540-lib-modules\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:47.087952 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:47.087918 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/37d1e678-8d49-4a40-9553-d3ea84111540-podres\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:47.088158 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:47.087951 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d1e678-8d49-4a40-9553-d3ea84111540-sys\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:47.088158 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:47.087959 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/37d1e678-8d49-4a40-9553-d3ea84111540-proc\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:47.088158 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:47.088016 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37d1e678-8d49-4a40-9553-d3ea84111540-lib-modules\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:47.097918 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:47.097864 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74sd5\" (UniqueName: \"kubernetes.io/projected/37d1e678-8d49-4a40-9553-d3ea84111540-kube-api-access-74sd5\") pod \"perf-node-gather-daemonset-d44kw\" (UID: \"37d1e678-8d49-4a40-9553-d3ea84111540\") " pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:47.194404 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:47.194317 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:47.324106 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:47.324082 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw"] Jul 09 15:02:47.325990 ip-10-0-136-93 kubenswrapper[2582]: W0709 15:02:47.325966 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod37d1e678_8d49_4a40_9553_d3ea84111540.slice/crio-6a8a623998138a002f885dc31ff2e3676d02d47a5c402042c66dfd008d8cbd70 WatchSource:0}: Error finding container 6a8a623998138a002f885dc31ff2e3676d02d47a5c402042c66dfd008d8cbd70: Status 404 returned error can't find the container with id 6a8a623998138a002f885dc31ff2e3676d02d47a5c402042c66dfd008d8cbd70 Jul 09 15:02:47.979833 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:47.979809 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7w4gm_3599785b-ae2d-4391-afb5-b6b50594143c/dns/0.log" Jul 09 15:02:48.006077 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:48.006052 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7w4gm_3599785b-ae2d-4391-afb5-b6b50594143c/kube-rbac-proxy/0.log" Jul 09 15:02:48.119949 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:48.119925 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n9hkg_9205fffb-e805-4302-8d1e-535e1874e263/dns-node-resolver/0.log" Jul 09 15:02:48.198858 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:48.198818 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" event={"ID":"37d1e678-8d49-4a40-9553-d3ea84111540","Type":"ContainerStarted","Data":"c54cb2eb42e5f2cc4615ddd0ef2b6294ae31a3c10cd0fdd8d70aa9c82189355a"} Jul 09 15:02:48.199075 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:48.198913 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:48.199075 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:48.198936 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" event={"ID":"37d1e678-8d49-4a40-9553-d3ea84111540","Type":"ContainerStarted","Data":"6a8a623998138a002f885dc31ff2e3676d02d47a5c402042c66dfd008d8cbd70"} Jul 09 15:02:48.220010 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:48.219960 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" podStartSLOduration=2.219942248 podStartE2EDuration="2.219942248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:48.21724536 +0000 UTC m=+2518.583339746" watchObservedRunningTime="2026-07-09 15:02:48.219942248 +0000 UTC m=+2518.586036633" Jul 09 15:02:48.716132 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:48.716102 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-bf6c9b6b-gh4ns_e4968b8a-08db-43ce-8c1f-0a53a5ad164b/registry/0.log" Jul 09 15:02:48.764073 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:48.764047 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ffxnv_1e64c211-f85a-4e12-8034-67a6c1e1bb88/node-ca/0.log" Jul 09 15:02:49.879175 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:49.879143 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77975789f4-69c6k_fa562d9c-ec3c-40b5-925f-a6862eab8006/kube-auth-proxy/0.log" Jul 09 15:02:50.030798 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:50.030771 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-stddl_4cdb3207-2563-400d-9dcc-043597f8e0e0/payload-pre-processing/0.log" Jul 09 15:02:50.061012 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:50.060974 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-xhrkm_633c7032-c50e-4182-bd2d-fee0d920768e/payload-processing/0.log" Jul 09 15:02:50.861358 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:50.861320 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ppm58_c405f127-dba7-4bb3-a573-a580ef9c9ad3/serve-healthcheck-canary/0.log" Jul 09 15:02:51.370215 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:51.370190 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ddv6d_ca7b3983-3431-45cb-a7b5-595e78ee3da4/kube-rbac-proxy/0.log" Jul 09 15:02:51.392670 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:51.392646 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ddv6d_ca7b3983-3431-45cb-a7b5-595e78ee3da4/exporter/0.log" Jul 09 15:02:51.416438 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:51.416412 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ddv6d_ca7b3983-3431-45cb-a7b5-595e78ee3da4/extractor/0.log" Jul 09 15:02:53.804792 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:53.804762 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tpgz5_980c62dc-ce17-4ac3-b661-e9a4faefdbc2/manager/0.log" Jul 09 15:02:53.928132 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:53.928101 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-95czt_0e7819cf-6cc4-4597-8e34-b5fb8a86101a/manager/0.log" Jul 09 15:02:54.039375 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:54.039349 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4fwdr_59035872-ab43-47f9-bcfe-5e1bbb034078/server/0.log" Jul 09 15:02:54.107281 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:54.107255 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-95rmq_4536ea27-ba7e-41f2-acbd-d9dac53ce64c/manager/0.log" Jul 09 15:02:54.178969 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:54.178940 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7757d956-5t57r_3568bfcb-c3d3-46ff-bb99-7d975be2d2b7/manager/0.log" Jul 09 15:02:54.198092 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:54.198071 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7757d956-5t57r_3568bfcb-c3d3-46ff-bb99-7d975be2d2b7/copy-manifests/0.log" Jul 09 15:02:54.213832 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:54.213807 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ltzpn/perf-node-gather-daemonset-d44kw" Jul 09 15:02:55.195994 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:55.195969 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77975789f4-69c6k_fa562d9c-ec3c-40b5-925f-a6862eab8006/kube-auth-proxy/0.log" Jul 09 15:02:55.313522 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:55.313496 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-stddl_4cdb3207-2563-400d-9dcc-043597f8e0e0/payload-pre-processing/0.log" Jul 09 15:02:55.338389 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:55.338360 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-xhrkm_633c7032-c50e-4182-bd2d-fee0d920768e/payload-processing/0.log" Jul 09 15:02:56.865377 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:56.865351 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77975789f4-69c6k_fa562d9c-ec3c-40b5-925f-a6862eab8006/kube-auth-proxy/0.log" Jul 09 15:02:56.994030 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:56.993994 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-stddl_4cdb3207-2563-400d-9dcc-043597f8e0e0/payload-pre-processing/0.log" Jul 09 15:02:57.016157 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:02:57.016128 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-xhrkm_633c7032-c50e-4182-bd2d-fee0d920768e/payload-processing/0.log" Jul 09 15:03:02.285883 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:02.285849 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz8cc_2a720bc8-0c99-4249-801c-a6d8e49ff8f3/kube-multus-additional-cni-plugins/0.log" Jul 09 15:03:02.309366 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:02.309336 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz8cc_2a720bc8-0c99-4249-801c-a6d8e49ff8f3/egress-router-binary-copy/0.log" Jul 09 15:03:02.334891 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:02.334843 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz8cc_2a720bc8-0c99-4249-801c-a6d8e49ff8f3/cni-plugins/0.log" Jul 09 15:03:02.359176 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:02.359150 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz8cc_2a720bc8-0c99-4249-801c-a6d8e49ff8f3/bond-cni-plugin/0.log" Jul 09 15:03:02.382665 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:02.382641 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz8cc_2a720bc8-0c99-4249-801c-a6d8e49ff8f3/routeoverride-cni/0.log" Jul 09 15:03:02.407662 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:02.407633 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz8cc_2a720bc8-0c99-4249-801c-a6d8e49ff8f3/whereabouts-cni-bincopy/0.log" Jul 09 15:03:02.431623 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:02.431601 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dz8cc_2a720bc8-0c99-4249-801c-a6d8e49ff8f3/whereabouts-cni/0.log" Jul 09 15:03:02.818163 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:02.818135 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bjpmw_45141af1-065c-4341-91be-37adc932e731/kube-multus/0.log" Jul 09 15:03:03.013279 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:03.013202 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vkk5f_6050faa0-7272-4fc1-8392-6f072ab52c5a/network-metrics-daemon/0.log" Jul 09 15:03:03.036080 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:03.036057 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vkk5f_6050faa0-7272-4fc1-8392-6f072ab52c5a/kube-rbac-proxy/0.log" Jul 09 15:03:04.112911 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:04.112850 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5qkd_b5e0a3b3-6bbc-4c39-9b01-035342233924/ovn-controller/0.log" Jul 09 15:03:04.134294 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:04.134270 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5qkd_b5e0a3b3-6bbc-4c39-9b01-035342233924/ovn-acl-logging/0.log" Jul 09 15:03:04.159958 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:04.159931 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5qkd_b5e0a3b3-6bbc-4c39-9b01-035342233924/kube-rbac-proxy-node/0.log" Jul 09 15:03:04.186203 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:04.186180 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5qkd_b5e0a3b3-6bbc-4c39-9b01-035342233924/kube-rbac-proxy-ovn-metrics/0.log" Jul 09 15:03:04.210407 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:04.210384 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5qkd_b5e0a3b3-6bbc-4c39-9b01-035342233924/northd/0.log" Jul 09 15:03:04.236552 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:04.236516 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5qkd_b5e0a3b3-6bbc-4c39-9b01-035342233924/nbdb/0.log" Jul 09 15:03:04.260099 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:04.260074 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5qkd_b5e0a3b3-6bbc-4c39-9b01-035342233924/sbdb/0.log" Jul 09 15:03:04.312221 ip-10-0-136-93 kubenswrapper[2582]: I0709 15:03:04.312194 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l5qkd_b5e0a3b3-6bbc-4c39-9b01-035342233924/ovnkube-controller/0.log"