Jul 10 20:55:41.513222 ip-10-0-131-227 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 10 20:55:41.513233 ip-10-0-131-227 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 10 20:55:41.513240 ip-10-0-131-227 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 10 20:55:41.513442 ip-10-0-131-227 systemd[1]: Failed to start Kubernetes Kubelet. Jul 10 20:55:51.601454 ip-10-0-131-227 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 10 20:55:51.601476 ip-10-0-131-227 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 78f6eeacfafb415888642372180e5fe4 -- Jul 10 20:57:45.237647 ip-10-0-131-227 systemd[1]: Starting Kubernetes Kubelet... Jul 10 20:57:46.743346 ip-10-0-131-227 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 20:57:46.743346 ip-10-0-131-227 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 20:57:46.743346 ip-10-0-131-227 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 20:57:46.743346 ip-10-0-131-227 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 20:57:46.743346 ip-10-0-131-227 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 20:57:46.744444 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.744347 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 20:57:46.748280 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748265 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:57:46.748280 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748280 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748284 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748288 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748291 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748294 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748297 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748300 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748303 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748305 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748308 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748315 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748318 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748321 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748323 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748326 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748328 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748331 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748333 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748336 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748338 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:57:46.748334 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748341 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748344 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748347 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748350 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748352 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748355 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748358 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748361 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748363 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748366 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748369 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748371 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748374 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748376 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748379 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748381 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748384 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748386 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748388 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:57:46.748821 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748405 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748410 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748414 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748418 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748422 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748426 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748430 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748433 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748435 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748438 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748440 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748444 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748446 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748449 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748452 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748456 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748459 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748462 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748465 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748468 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:57:46.749288 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748470 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748473 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748475 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748479 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748483 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748486 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748489 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748492 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748495 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748497 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748501 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748503 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748506 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748508 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748511 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748514 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748516 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748519 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748521 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:57:46.749785 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748524 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748526 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748529 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748531 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748534 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748537 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748540 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748542 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748546 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748550 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748552 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748555 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748557 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748560 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748563 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748565 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748568 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748570 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748574 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.748578 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:57:46.750221 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749255 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749262 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749266 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749268 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749271 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749274 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749277 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749280 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749283 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749286 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749288 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749291 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749293 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749296 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749299 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749302 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749304 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749307 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749310 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749313 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:57:46.750743 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749316 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749318 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749321 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749323 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749326 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749328 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749331 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749334 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749336 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749339 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749341 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749344 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749346 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749349 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749351 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749354 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749356 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749358 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749361 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749364 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:57:46.751219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749366 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749370 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749374 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749376 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749379 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749382 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.749385 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750173 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750180 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750183 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750188 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750191 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750194 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750197 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750199 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750202 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750204 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750207 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750209 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:57:46.751732 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750212 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750216 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750220 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750223 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750226 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750229 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750233 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750235 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750238 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750242 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750246 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750249 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750252 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750255 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750258 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750260 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750263 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750266 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750269 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:57:46.752189 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750273 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750275 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750278 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750281 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750286 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750289 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750292 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750295 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750298 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750301 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750304 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750307 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750310 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750312 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750315 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750317 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750319 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750322 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750325 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750327 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:57:46.752665 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750330 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750728 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750733 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750737 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750740 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750743 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750747 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750751 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750754 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750758 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750760 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750765 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750769 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750773 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750776 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750778 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750781 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750783 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750786 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:57:46.753138 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750789 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750791 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750794 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750797 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750800 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750802 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750805 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750807 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750810 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750812 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750815 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750817 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750820 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750822 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750825 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750829 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750832 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750835 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750838 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750842 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:57:46.753605 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750845 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750848 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750850 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750853 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750857 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750860 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750863 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750865 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750868 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750870 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750873 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750876 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750879 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750881 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750884 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750886 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750889 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750892 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750894 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750896 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:57:46.754093 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750899 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750901 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750904 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750907 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750909 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750912 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750915 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750917 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750920 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750922 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750925 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750927 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750930 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750933 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750935 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750937 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750942 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750944 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750947 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:57:46.754594 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750949 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750952 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750954 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750957 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750959 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750962 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750964 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750967 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750969 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750971 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750974 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750976 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750979 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750981 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750984 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750986 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750989 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750992 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750994 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:57:46.755060 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750997 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.750999 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751001 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751068 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751076 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751084 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751090 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751097 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751102 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751108 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751117 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751122 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751126 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751129 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751132 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751135 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751138 2566 flags.go:64] FLAG: --cgroup-root="" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751141 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751143 2566 flags.go:64] FLAG: --client-ca-file="" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751146 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751149 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751154 2566 flags.go:64] FLAG: --cluster-domain="" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751157 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751160 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 20:57:46.755511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751163 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751166 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751170 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751173 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751176 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751180 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751183 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751186 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751188 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751192 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751194 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751198 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751201 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751204 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751207 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751210 2566 flags.go:64] FLAG: --enable-server="true" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751213 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751218 2566 flags.go:64] FLAG: --event-burst="100" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751222 2566 flags.go:64] FLAG: --event-qps="50" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751225 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751228 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751231 2566 flags.go:64] FLAG: --eviction-hard="" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751235 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751238 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751241 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 20:57:46.756074 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751244 2566 flags.go:64] FLAG: --eviction-soft="" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751247 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751249 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751252 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751255 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751259 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751261 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751264 2566 flags.go:64] FLAG: --feature-gates="" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751268 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751271 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751274 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751277 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751280 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751283 2566 flags.go:64] FLAG: --help="false" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751285 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-131-227.ec2.internal" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751289 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751291 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751294 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751297 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751300 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751303 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751306 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751309 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 20:57:46.756675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751312 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751316 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751320 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751323 2566 flags.go:64] FLAG: --kube-reserved="" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751326 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751328 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751331 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751334 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751337 2566 flags.go:64] FLAG: --lock-file="" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751340 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751343 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751346 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751351 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751354 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751357 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751360 2566 flags.go:64] FLAG: --logging-format="text" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751362 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751366 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751369 2566 flags.go:64] FLAG: --manifest-url="" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751372 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751376 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751381 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751385 2566 flags.go:64] FLAG: --max-pods="110" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751388 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751391 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 20:57:46.757242 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751404 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751408 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751411 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751414 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751417 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751423 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751426 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751429 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751432 2566 flags.go:64] FLAG: --pod-cidr="" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751437 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751442 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751445 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751449 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751451 2566 flags.go:64] FLAG: --port="10250" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751454 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751457 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-026289fe54248d779" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751461 2566 flags.go:64] FLAG: --qos-reserved="" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751464 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751467 2566 flags.go:64] FLAG: --register-node="true" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751470 2566 flags.go:64] FLAG: --register-with-taints="" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751473 2566 flags.go:64] FLAG: --registry-burst="10" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751476 2566 flags.go:64] FLAG: --registry-qps="5" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751479 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751482 2566 flags.go:64] FLAG: --reserved-memory="" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751486 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 20:57:46.757874 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751489 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751492 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751494 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751498 2566 flags.go:64] FLAG: --runonce="false" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751501 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751504 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751507 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751510 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751513 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751515 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751518 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751521 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751524 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751527 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751530 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751533 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751536 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751540 2566 flags.go:64] FLAG: --system-cgroups="" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751543 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751548 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751551 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751553 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751558 2566 flags.go:64] FLAG: --tls-min-version="" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751561 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751563 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 20:57:46.758493 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751566 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751569 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751572 2566 flags.go:64] FLAG: --v="2" Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751576 2566 flags.go:64] FLAG: --version="false" Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751580 2566 flags.go:64] FLAG: --vmodule="" Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751584 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.751587 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751718 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751722 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751725 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751729 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751732 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751734 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751737 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751740 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751743 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751745 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751748 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751751 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751753 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751756 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751759 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:57:46.759103 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751761 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751763 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751767 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751770 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751772 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751775 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751777 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751780 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751782 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751785 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751788 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751792 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751795 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751798 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751800 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751803 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751805 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751808 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751810 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:57:46.759661 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751813 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751817 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751820 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751822 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751825 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751827 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751829 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751832 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751835 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751837 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751840 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751842 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751845 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751847 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751850 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751852 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751855 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751858 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751860 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751863 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:57:46.760107 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751866 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751868 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751871 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751873 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751876 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751878 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751881 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751883 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751886 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751888 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751891 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751893 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751896 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751899 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751903 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751907 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751910 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751912 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751915 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:57:46.760604 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751917 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751919 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751922 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751924 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751927 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751931 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751935 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751938 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751941 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751944 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751947 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751951 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751954 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751956 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751959 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751961 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751964 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751966 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751969 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751971 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:57:46.761119 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751974 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751976 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751978 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751981 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751983 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.751986 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.752797 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.759830 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.759846 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.760881 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.760888 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.760965 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.760971 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.760975 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.760978 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:57:46.761641 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.760981 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.760984 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.760987 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.760990 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.760993 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.760996 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.760998 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761001 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761010 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761013 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761016 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761018 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761021 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761024 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761027 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761031 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761035 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761038 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761041 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761044 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:57:46.762053 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761047 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761050 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761052 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761055 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761058 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761061 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761064 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761067 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761069 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761072 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761075 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761077 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761080 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761082 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761085 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761087 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761094 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761097 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761099 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761102 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:57:46.762567 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761104 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761117 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761120 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761122 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761125 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761128 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761131 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761133 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761136 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761138 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761141 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761143 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761146 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761148 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761151 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761153 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761156 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761160 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761162 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761165 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:57:46.763029 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761167 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761170 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761173 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761175 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761178 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761180 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761182 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761185 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761187 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761190 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761192 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761195 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761197 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761200 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761207 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761210 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761213 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761215 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761218 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761220 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:57:46.763526 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761223 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:57:46.763992 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761225 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:57:46.763992 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761227 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:57:46.763992 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761232 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:57:46.763992 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761235 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:57:46.763992 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761238 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:57:46.763992 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761242 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:57:46.763992 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761246 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:57:46.763992 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761248 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:57:46.763992 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761252 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:57:46.763992 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761255 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:57:46.763992 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761257 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:57:46.763992 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761260 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:57:46.763992 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761262 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:57:46.763992 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761265 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.761270 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761469 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761474 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761477 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761480 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761483 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761486 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761489 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761492 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761495 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761497 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761499 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761508 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761511 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:57:46.764324 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761513 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761515 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761518 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761520 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761523 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761526 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761529 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761531 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761533 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761536 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761538 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761541 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761543 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761546 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761548 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761551 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761553 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761556 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761558 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761561 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:57:46.764722 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761563 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761566 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761568 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761571 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761573 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761576 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761578 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761580 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761583 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761585 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761588 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761596 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761599 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761601 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761604 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761613 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761617 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761620 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761622 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:57:46.765214 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761624 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761627 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761629 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761632 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761634 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761637 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761640 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761642 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761645 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761648 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761650 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761653 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761656 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761658 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761661 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761663 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761666 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761668 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761671 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761673 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:57:46.765686 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761675 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761678 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761680 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761683 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761685 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761695 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761699 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761702 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761705 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761708 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761711 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761713 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761717 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761720 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761722 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761725 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761727 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761730 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761733 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:57:46.766148 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761736 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:57:46.766623 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761738 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:57:46.766623 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761741 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:57:46.766623 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761743 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:57:46.766623 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761746 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:57:46.766623 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761748 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:57:46.766623 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761751 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:57:46.766623 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:46.761753 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:57:46.766623 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.761757 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 20:57:46.766623 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.761873 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 20:57:46.766623 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.764567 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 20:57:46.766623 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.765973 2566 server.go:1019] "Starting client certificate rotation" Jul 10 20:57:46.766623 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.766077 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 20:57:46.766623 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.766118 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 20:57:46.768714 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.768699 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 20:57:46.772852 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.770434 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 20:57:46.785255 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.785234 2566 log.go:25] "Validated CRI v1 runtime API" Jul 10 20:57:46.787186 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.787175 2566 log.go:25] "Validated CRI v1 image API" Jul 10 20:57:46.789431 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.789417 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 20:57:46.793453 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.793437 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 20:57:46.793809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.793787 2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8d32fadc-edcf-4358-9b55-a61a20dee4db:/dev/nvme0n1p3 9ec8f1a5-7089-462e-ab1e-d6d83a09368d:/dev/nvme0n1p4] Jul 10 20:57:46.793888 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.793810 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 20:57:46.799203 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.799090 2566 manager.go:216] Machine: {Timestamp:2026-07-10 20:57:46.797511978 +0000 UTC m=+0.381207571 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099091 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a352c37741a2ffecbf718764c7237 SystemUUID:ec2a352c-3774-1a2f-fecb-f718764c7237 BootID:78f6eeac-fafb-4158-8864-2372180e5fe4 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fd:0c:48:62:65 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fd:0c:48:62:65 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9a:5a:04:d0:36:fe Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 20:57:46.799203 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.799194 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 20:57:46.799354 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.799294 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 20:57:46.802602 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.802576 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 20:57:46.802735 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.802605 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-227.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 20:57:46.804217 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.804207 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 20:57:46.804256 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.804220 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 20:57:46.804256 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.804232 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 20:57:46.804256 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.804243 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 20:57:46.805511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.805501 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 20:57:46.808475 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.808463 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 10 20:57:46.808602 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.808592 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 20:57:46.808642 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.808623 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 10 20:57:46.808642 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.808632 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 20:57:46.808696 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.808646 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 20:57:46.808696 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.808655 2566 kubelet.go:398] "Adding apiserver pod source" Jul 10 20:57:46.808696 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.808667 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 20:57:46.809131 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.809117 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 20:57:46.809171 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.809151 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 20:57:46.810930 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.810912 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d8x28" Jul 10 20:57:46.812883 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.812865 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 20:57:46.814849 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.814835 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 20:57:46.814896 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.814866 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 20:57:46.815055 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.815035 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 20:57:46.815055 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.815052 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 20:57:46.815055 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.815063 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 20:57:46.815204 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.815071 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 20:57:46.815204 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.815081 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 20:57:46.815204 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.815087 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 20:57:46.815907 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.815897 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 20:57:46.815907 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.815907 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 20:57:46.815991 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.815939 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 20:57:46.815991 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.815952 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 20:57:46.815991 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.815965 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 20:57:46.815991 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.815973 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 20:57:46.817004 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.816988 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 20:57:46.817004 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.816999 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 20:57:46.817901 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:46.817880 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 20:57:46.817958 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:46.817907 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-227.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 20:57:46.819286 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.819271 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-227.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 20:57:46.820283 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.820264 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d8x28" Jul 10 20:57:46.820409 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.820386 2566 server.go:1268] "Started kubelet" Jul 10 20:57:46.820511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.820479 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 20:57:46.820572 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.820471 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 20:57:46.820572 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.820542 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 20:57:46.824444 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.824423 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 20:57:46.824666 ip-10-0-131-227 systemd[1]: Started Kubernetes Kubelet. Jul 10 20:57:46.828540 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:46.828517 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 20:57:46.828540 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.828535 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 10 20:57:46.831867 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.831850 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 20:57:46.831951 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.831866 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 20:57:46.831951 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:46.831935 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jul 10 20:57:46.832430 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.832388 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 20:57:46.832430 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.832390 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 20:57:46.832590 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.832442 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 20:57:46.832590 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.832500 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 10 20:57:46.832590 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.832508 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 10 20:57:46.832716 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.832653 2566 factory.go:55] Registering systemd factory Jul 10 20:57:46.832716 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.832669 2566 factory.go:223] Registration of the systemd container factory successfully Jul 10 20:57:46.832990 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.832973 2566 factory.go:153] Registering CRI-O factory Jul 10 20:57:46.832990 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.832991 2566 factory.go:223] Registration of the crio container factory successfully Jul 10 20:57:46.833119 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.833040 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 20:57:46.833119 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.833062 2566 factory.go:103] Registering Raw factory Jul 10 20:57:46.833119 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.833075 2566 manager.go:1196] Started watching for new ooms in manager Jul 10 20:57:46.833625 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.833608 2566 manager.go:349] Starting recovery of all containers Jul 10 20:57:46.836077 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.836052 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:57:46.838180 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:46.838160 2566 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-227.ec2.internal\" not found" node="ip-10-0-131-227.ec2.internal" Jul 10 20:57:46.843916 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.843811 2566 manager.go:354] Recovery completed Jul 10 20:57:46.845160 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:46.845128 2566 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 10 20:57:46.848054 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.848043 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:57:46.850083 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.850069 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:57:46.850161 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.850100 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:57:46.850161 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.850115 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:57:46.850688 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.850675 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 20:57:46.850688 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.850687 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 20:57:46.850803 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.850703 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 10 20:57:46.850803 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.850725 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:57:46.851450 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.851431 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:57:46.851555 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.851466 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:57:46.851555 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.851480 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:57:46.851980 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.851968 2566 policy_none.go:49] "None policy: Start" Jul 10 20:57:46.852021 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.851984 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 20:57:46.852021 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.851997 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 20:57:46.853051 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.853038 2566 policy_none.go:47] "Start" Jul 10 20:57:46.892792 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.892778 2566 manager.go:337] "Starting Device Plugin manager" Jul 10 20:57:46.892863 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:46.892810 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 20:57:46.892863 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.892820 2566 server.go:87] "Starting device plugin registration server" Jul 10 20:57:46.893092 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.893080 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 20:57:46.893129 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.893093 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 20:57:46.893199 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.893181 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 20:57:46.893285 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.893266 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 20:57:46.893285 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.893281 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 20:57:46.893886 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:46.893872 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 20:57:46.893947 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:46.893913 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-227.ec2.internal\" not found" Jul 10 20:57:46.993690 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.993622 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:57:46.995288 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.994870 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:57:46.995288 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.994906 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:57:46.995288 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.994920 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:57:46.995288 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:46.994948 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.005157 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.005137 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.005209 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:47.005160 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-227.ec2.internal\": node \"ip-10-0-131-227.ec2.internal\" not found" Jul 10 20:57:47.040441 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:47.040410 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jul 10 20:57:47.060312 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.060281 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 20:57:47.061428 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.061386 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 20:57:47.061519 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.061455 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 20:57:47.061519 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.061485 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 20:57:47.061598 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:47.061527 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 20:57:47.063909 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.063890 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:57:47.140739 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:47.140694 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jul 10 20:57:47.161897 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.161867 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal"] Jul 10 20:57:47.161984 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.161950 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:57:47.163677 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.163663 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:57:47.163755 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.163692 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:57:47.163755 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.163702 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:57:47.164868 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.164856 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:57:47.164990 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.164977 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.165024 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.165010 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:57:47.165616 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.165600 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:57:47.165701 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.165632 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:57:47.165701 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.165600 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:57:47.165701 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.165665 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:57:47.165701 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.165676 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:57:47.165701 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.165648 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:57:47.166727 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.166714 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.166787 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.166740 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:57:47.167653 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.167641 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:57:47.167719 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.167667 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:57:47.167719 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.167682 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:57:47.180961 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:47.180941 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-227.ec2.internal\" not found" node="ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.185239 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:47.185225 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-227.ec2.internal\" not found" node="ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.235509 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.235482 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5275d30bb107dec4f3d86b0471c4d693-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal\" (UID: \"5275d30bb107dec4f3d86b0471c4d693\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.235614 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.235512 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5275d30bb107dec4f3d86b0471c4d693-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal\" (UID: \"5275d30bb107dec4f3d86b0471c4d693\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.235614 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.235529 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1cad2d201741314c351e55ceddd77eeb-config\") pod \"kube-apiserver-proxy-ip-10-0-131-227.ec2.internal\" (UID: \"1cad2d201741314c351e55ceddd77eeb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.241573 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:47.241554 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jul 10 20:57:47.335869 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.335803 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5275d30bb107dec4f3d86b0471c4d693-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal\" (UID: \"5275d30bb107dec4f3d86b0471c4d693\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.335869 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.335850 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5275d30bb107dec4f3d86b0471c4d693-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal\" (UID: \"5275d30bb107dec4f3d86b0471c4d693\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.335957 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.335876 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1cad2d201741314c351e55ceddd77eeb-config\") pod \"kube-apiserver-proxy-ip-10-0-131-227.ec2.internal\" (UID: \"1cad2d201741314c351e55ceddd77eeb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.335957 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.335887 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5275d30bb107dec4f3d86b0471c4d693-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal\" (UID: \"5275d30bb107dec4f3d86b0471c4d693\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.335957 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.335912 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1cad2d201741314c351e55ceddd77eeb-config\") pod \"kube-apiserver-proxy-ip-10-0-131-227.ec2.internal\" (UID: \"1cad2d201741314c351e55ceddd77eeb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.336046 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.335980 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5275d30bb107dec4f3d86b0471c4d693-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal\" (UID: \"5275d30bb107dec4f3d86b0471c4d693\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.341926 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:47.341908 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jul 10 20:57:47.442463 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:47.442436 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jul 10 20:57:47.482616 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.482590 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.488533 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.488515 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" Jul 10 20:57:47.543322 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:47.543291 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jul 10 20:57:47.643817 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:47.643757 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jul 10 20:57:47.744265 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:47.744230 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jul 10 20:57:47.765442 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.765415 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 20:57:47.765567 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.765551 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 20:57:47.765606 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.765584 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 20:57:47.828670 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.828630 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 20:52:46 +0000 UTC" deadline="2027-12-04 11:26:00.510485275 +0000 UTC" Jul 10 20:57:47.828670 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.828666 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12278h28m12.681823022s" Jul 10 20:57:47.844556 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:47.844536 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jul 10 20:57:47.895951 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.895871 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:57:47.945011 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:47.944976 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jul 10 20:57:47.946304 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:47.946279 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cad2d201741314c351e55ceddd77eeb.slice/crio-cf9e6cff479ce835ab1c9bf4634eb1d38ab68bac1abb7806115e607db131c0bd WatchSource:0}: Error finding container cf9e6cff479ce835ab1c9bf4634eb1d38ab68bac1abb7806115e607db131c0bd: Status 404 returned error can't find the container with id cf9e6cff479ce835ab1c9bf4634eb1d38ab68bac1abb7806115e607db131c0bd Jul 10 20:57:47.946461 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:47.946440 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5275d30bb107dec4f3d86b0471c4d693.slice/crio-f7b9b03e3aa1f912be3d90fa489c19c5c3abd1ae6f39420501391b28b4034db0 WatchSource:0}: Error finding container f7b9b03e3aa1f912be3d90fa489c19c5c3abd1ae6f39420501391b28b4034db0: Status 404 returned error can't find the container with id f7b9b03e3aa1f912be3d90fa489c19c5c3abd1ae6f39420501391b28b4034db0 Jul 10 20:57:47.952409 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:47.952373 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 20:57:48.045696 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:48.045653 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-227.ec2.internal\" not found" Jul 10 20:57:48.064934 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.064890 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" event={"ID":"1cad2d201741314c351e55ceddd77eeb","Type":"ContainerStarted","Data":"cf9e6cff479ce835ab1c9bf4634eb1d38ab68bac1abb7806115e607db131c0bd"} Jul 10 20:57:48.065908 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.065886 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" event={"ID":"5275d30bb107dec4f3d86b0471c4d693","Type":"ContainerStarted","Data":"f7b9b03e3aa1f912be3d90fa489c19c5c3abd1ae6f39420501391b28b4034db0"} Jul 10 20:57:48.138407 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.138380 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:57:48.232262 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.232179 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" Jul 10 20:57:48.241798 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.241771 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 20:57:48.243025 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.243011 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" Jul 10 20:57:48.253421 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.253389 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 20:57:48.575028 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.574926 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:57:48.604111 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.604056 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:57:48.810765 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.810732 2566 apiserver.go:52] "Watching apiserver" Jul 10 20:57:48.817232 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.817203 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 20:57:48.817701 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.817673 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-9788m","kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf","openshift-dns/node-resolver-62r4h","openshift-image-registry/node-ca-rnz7t","openshift-multus/multus-6kpsn","openshift-multus/multus-additional-cni-plugins-kv7p8","openshift-multus/network-metrics-daemon-lk7sj","kube-system/konnectivity-agent-nmqk9","openshift-cluster-node-tuning-operator/tuned-t5v8m","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal","openshift-network-diagnostics/network-check-target-8lb77","openshift-network-operator/iptables-alerter-282w9","openshift-ovn-kubernetes/ovnkube-node-j5q89"] Jul 10 20:57:48.821500 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.820788 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.823226 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.823196 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-282w9" Jul 10 20:57:48.824453 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.824431 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 20:57:48.825469 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.824856 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-945rn\"" Jul 10 20:57:48.825469 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.824928 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 20:57:48.825469 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.825193 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 20:57:48.825469 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.825233 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 20:57:48.826211 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.826125 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 20:57:48.826211 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.826167 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 20:57:48.826494 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.826457 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4njcv\"" Jul 10 20:57:48.826596 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.826543 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 20:57:48.826656 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.826635 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 20:57:48.829036 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.828324 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.829911 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.829893 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-62r4h" Jul 10 20:57:48.830585 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.830379 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jksst\"" Jul 10 20:57:48.830585 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.830387 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 20:57:48.831376 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.831065 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 20:57:48.831376 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.831351 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 20:57:48.832329 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.832164 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-82m9p\"" Jul 10 20:57:48.832329 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.832210 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 20:57:48.832498 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.832473 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 20:57:48.832498 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.832484 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rnz7t" Jul 10 20:57:48.833131 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.833110 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 20:57:48.833829 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.833812 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.834922 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.834903 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nmqk9" Jul 10 20:57:48.835520 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.835500 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 20:57:48.835771 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.835753 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 20:57:48.836232 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.836217 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 20:57:48.837690 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.836569 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7k9jx\"" Jul 10 20:57:48.837690 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.836636 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kch8d\"" Jul 10 20:57:48.837690 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.836706 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 20:57:48.837690 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.837384 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gqv85\"" Jul 10 20:57:48.837690 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.837476 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 20:57:48.837690 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.837604 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 20:57:48.838290 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.838260 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:48.838495 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:48.838469 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:57:48.839821 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.839804 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.841295 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.841275 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.842315 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.842298 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 20:57:48.842625 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.842604 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5kgwf\"" Jul 10 20:57:48.842885 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.842845 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:48.843089 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:48.843069 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:57:48.843250 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.843207 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 20:57:48.843419 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.843384 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 20:57:48.844085 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.843788 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 20:57:48.844085 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.843924 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:57:48.844085 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.843926 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 20:57:48.844085 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844037 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 20:57:48.844085 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844044 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 20:57:48.844353 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:48.844102 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:57:48.844353 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844297 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 20:57:48.844353 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844304 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-registration-dir\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.844353 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844336 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-multus-cni-dir\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.844573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844360 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-cnibin\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.844573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844419 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-system-cni-dir\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.844573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844376 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5xp4g\"" Jul 10 20:57:48.844573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844446 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-os-release\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.844573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844473 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgqfc\" (UniqueName: \"kubernetes.io/projected/3ba53ae5-7a20-4763-a164-0924edc26ba2-kube-api-access-sgqfc\") pod \"node-resolver-62r4h\" (UID: \"3ba53ae5-7a20-4763-a164-0924edc26ba2\") " pod="openshift-dns/node-resolver-62r4h" Jul 10 20:57:48.844573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844499 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3fc26a94-4249-49e4-af9c-f4deea65bc50-multus-daemon-config\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.844573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844520 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-cnibin\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.844573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844542 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-cni-binary-copy\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.844573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844566 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.844995 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844586 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e973bb49-d732-4f0d-b518-7f25d5ebd9ff-host\") pod \"node-ca-rnz7t\" (UID: \"e973bb49-d732-4f0d-b518-7f25d5ebd9ff\") " pod="openshift-image-registry/node-ca-rnz7t" Jul 10 20:57:48.844995 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844654 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwsjw\" (UniqueName: \"kubernetes.io/projected/e973bb49-d732-4f0d-b518-7f25d5ebd9ff-kube-api-access-dwsjw\") pod \"node-ca-rnz7t\" (UID: \"e973bb49-d732-4f0d-b518-7f25d5ebd9ff\") " pod="openshift-image-registry/node-ca-rnz7t" Jul 10 20:57:48.844995 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844700 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-os-release\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.844995 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844748 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-multus-socket-dir-parent\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.844995 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844783 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-etc-selinux\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.844995 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844807 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ba53ae5-7a20-4763-a164-0924edc26ba2-hosts-file\") pod \"node-resolver-62r4h\" (UID: \"3ba53ae5-7a20-4763-a164-0924edc26ba2\") " pod="openshift-dns/node-resolver-62r4h" Jul 10 20:57:48.844995 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-run-netns\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.844995 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844852 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-run-multus-certs\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.844995 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844876 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngsv4\" (UniqueName: \"kubernetes.io/projected/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-kube-api-access-ngsv4\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.844995 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844916 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-device-dir\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.844995 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844953 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-sys-fs\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.844995 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.844980 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zflc8\" (UniqueName: \"kubernetes.io/projected/e2e37f80-e59b-4aae-a1ef-bc808092cd22-kube-api-access-zflc8\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845013 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-var-lib-cni-multus\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845066 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3fc26a94-4249-49e4-af9c-f4deea65bc50-cni-binary-copy\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845094 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-multus-conf-dir\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845134 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhqnl\" (UniqueName: \"kubernetes.io/projected/50aacd38-b488-40dc-98c2-cad52a033e3e-kube-api-access-jhqnl\") pod \"iptables-alerter-282w9\" (UID: \"50aacd38-b488-40dc-98c2-cad52a033e3e\") " pod="openshift-network-operator/iptables-alerter-282w9" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845159 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e973bb49-d732-4f0d-b518-7f25d5ebd9ff-serviceca\") pod \"node-ca-rnz7t\" (UID: \"e973bb49-d732-4f0d-b518-7f25d5ebd9ff\") " pod="openshift-image-registry/node-ca-rnz7t" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845216 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bmh4\" (UniqueName: \"kubernetes.io/projected/3fc26a94-4249-49e4-af9c-f4deea65bc50-kube-api-access-8bmh4\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845241 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845276 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ba53ae5-7a20-4763-a164-0924edc26ba2-tmp-dir\") pod \"node-resolver-62r4h\" (UID: \"3ba53ae5-7a20-4763-a164-0924edc26ba2\") " pod="openshift-dns/node-resolver-62r4h" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845317 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-run-k8s-cni-cncf-io\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845351 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-var-lib-cni-bin\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845406 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-hostroot\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845434 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-etc-kubernetes\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845459 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845483 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/50aacd38-b488-40dc-98c2-cad52a033e3e-iptables-alerter-script\") pod \"iptables-alerter-282w9\" (UID: \"50aacd38-b488-40dc-98c2-cad52a033e3e\") " pod="openshift-network-operator/iptables-alerter-282w9" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845504 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-system-cni-dir\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.845573 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845526 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-var-lib-kubelet\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.846309 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845586 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50aacd38-b488-40dc-98c2-cad52a033e3e-host-slash\") pod \"iptables-alerter-282w9\" (UID: \"50aacd38-b488-40dc-98c2-cad52a033e3e\") " pod="openshift-network-operator/iptables-alerter-282w9" Jul 10 20:57:48.846309 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845624 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-kubelet-dir\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.846309 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.845647 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-socket-dir\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.852035 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.852010 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 20:57:48.869142 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.869123 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-74rzk" Jul 10 20:57:48.878359 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.878333 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-74rzk" Jul 10 20:57:48.933877 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.933852 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 20:57:48.945880 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.945852 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-var-lib-kubelet\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.946020 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.945891 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-host\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.946020 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.945922 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-run-multus-certs\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.946020 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.945947 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngsv4\" (UniqueName: \"kubernetes.io/projected/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-kube-api-access-ngsv4\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.946020 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.945971 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-device-dir\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.946020 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.945993 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-sys-fs\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.946020 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946018 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zflc8\" (UniqueName: \"kubernetes.io/projected/e2e37f80-e59b-4aae-a1ef-bc808092cd22-kube-api-access-zflc8\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.946318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946020 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-run-multus-certs\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.946318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946044 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-slash\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.946318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946067 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ec81adf8-e539-414a-9bde-d607f042b620-etc-tuned\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.946318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946091 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhqnl\" (UniqueName: \"kubernetes.io/projected/50aacd38-b488-40dc-98c2-cad52a033e3e-kube-api-access-jhqnl\") pod \"iptables-alerter-282w9\" (UID: \"50aacd38-b488-40dc-98c2-cad52a033e3e\") " pod="openshift-network-operator/iptables-alerter-282w9" Jul 10 20:57:48.946318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946093 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-device-dir\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.946318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946102 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-sys-fs\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.946318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946122 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-kubelet-config\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:48.946318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946181 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-systemd\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.946318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946233 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bmh4\" (UniqueName: \"kubernetes.io/projected/3fc26a94-4249-49e4-af9c-f4deea65bc50-kube-api-access-8bmh4\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.946318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946265 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:48.946318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946291 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-run-ovn\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946331 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-log-socket\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946357 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c1272af-f33d-4d09-9e09-bb846baccbc0-ovnkube-script-lib\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946381 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-sysctl-conf\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946425 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946466 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-kubelet\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946501 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-etc-openvswitch\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946531 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-run\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946567 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec81adf8-e539-414a-9bde-d607f042b620-tmp\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946595 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-var-lib-kubelet\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946619 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50aacd38-b488-40dc-98c2-cad52a033e3e-host-slash\") pod \"iptables-alerter-282w9\" (UID: \"50aacd38-b488-40dc-98c2-cad52a033e3e\") " pod="openshift-network-operator/iptables-alerter-282w9" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946643 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-kubelet-dir\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946669 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcjp8\" (UniqueName: \"kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8\") pod \"network-check-target-8lb77\" (UID: \"03703dd3-bccf-4be3-bd9d-68c093249f6c\") " pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946675 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-var-lib-kubelet\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946693 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-sys\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946718 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-cnibin\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946740 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rfnn\" (UniqueName: \"kubernetes.io/projected/ec81adf8-e539-414a-9bde-d607f042b620-kube-api-access-9rfnn\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.946929 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946768 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3fc26a94-4249-49e4-af9c-f4deea65bc50-multus-daemon-config\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946775 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50aacd38-b488-40dc-98c2-cad52a033e3e-host-slash\") pod \"iptables-alerter-282w9\" (UID: \"50aacd38-b488-40dc-98c2-cad52a033e3e\") " pod="openshift-network-operator/iptables-alerter-282w9" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946791 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-cni-binary-copy\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946818 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-cnibin\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946744 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-kubelet-dir\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946838 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e973bb49-d732-4f0d-b518-7f25d5ebd9ff-host\") pod \"node-ca-rnz7t\" (UID: \"e973bb49-d732-4f0d-b518-7f25d5ebd9ff\") " pod="openshift-image-registry/node-ca-rnz7t" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946865 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-systemd-units\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946891 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-multus-socket-dir-parent\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946915 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/336c95f4-fe3d-4af2-b2bf-395b7cae9633-agent-certs\") pod \"konnectivity-agent-nmqk9\" (UID: \"336c95f4-fe3d-4af2-b2bf-395b7cae9633\") " pod="kube-system/konnectivity-agent-nmqk9" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946937 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-cni-bin\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946961 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-modprobe-d\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.946986 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-run-netns\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947009 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-run-netns\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947032 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-run-systemd\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947048 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947075 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-run-openvswitch\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947104 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-run-ovn-kubernetes\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.947738 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947130 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947161 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-var-lib-cni-multus\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947133 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e973bb49-d732-4f0d-b518-7f25d5ebd9ff-host\") pod \"node-ca-rnz7t\" (UID: \"e973bb49-d732-4f0d-b518-7f25d5ebd9ff\") " pod="openshift-image-registry/node-ca-rnz7t" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947216 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-run-netns\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947212 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-var-lib-cni-multus\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947263 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-multus-socket-dir-parent\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947275 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3fc26a94-4249-49e4-af9c-f4deea65bc50-cni-binary-copy\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947289 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-cni-binary-copy\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947302 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-multus-conf-dir\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947336 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3fc26a94-4249-49e4-af9c-f4deea65bc50-multus-daemon-config\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947339 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e973bb49-d732-4f0d-b518-7f25d5ebd9ff-serviceca\") pod \"node-ca-rnz7t\" (UID: \"e973bb49-d732-4f0d-b518-7f25d5ebd9ff\") " pod="openshift-image-registry/node-ca-rnz7t" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947382 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-multus-conf-dir\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947383 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947491 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ba53ae5-7a20-4763-a164-0924edc26ba2-tmp-dir\") pod \"node-resolver-62r4h\" (UID: \"3ba53ae5-7a20-4763-a164-0924edc26ba2\") " pod="openshift-dns/node-resolver-62r4h" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947523 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-var-lib-openvswitch\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947529 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947563 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-lib-modules\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947801 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ba53ae5-7a20-4763-a164-0924edc26ba2-tmp-dir\") pod \"node-resolver-62r4h\" (UID: \"3ba53ae5-7a20-4763-a164-0924edc26ba2\") " pod="openshift-dns/node-resolver-62r4h" Jul 10 20:57:48.948531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947851 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-run-k8s-cni-cncf-io\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947877 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-var-lib-cni-bin\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947913 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-hostroot\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947923 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3fc26a94-4249-49e4-af9c-f4deea65bc50-cni-binary-copy\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947939 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-etc-kubernetes\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947947 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e973bb49-d732-4f0d-b518-7f25d5ebd9ff-serviceca\") pod \"node-ca-rnz7t\" (UID: \"e973bb49-d732-4f0d-b518-7f25d5ebd9ff\") " pod="openshift-image-registry/node-ca-rnz7t" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947963 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/50aacd38-b488-40dc-98c2-cad52a033e3e-iptables-alerter-script\") pod \"iptables-alerter-282w9\" (UID: \"50aacd38-b488-40dc-98c2-cad52a033e3e\") " pod="openshift-network-operator/iptables-alerter-282w9" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947984 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-var-lib-cni-bin\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.947994 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-hostroot\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948027 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs\") pod \"network-metrics-daemon-lk7sj\" (UID: \"ca5ea89b-dd36-4208-b892-a9b1176c0234\") " pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948048 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-etc-kubernetes\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948098 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-dbus\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948131 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-node-log\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948151 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-host-run-k8s-cni-cncf-io\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948179 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-system-cni-dir\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948204 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-socket-dir\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948228 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-registration-dir\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948234 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-system-cni-dir\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.949384 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-multus-cni-dir\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-system-cni-dir\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948286 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-registration-dir\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948303 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-os-release\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948327 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgqfc\" (UniqueName: \"kubernetes.io/projected/3ba53ae5-7a20-4763-a164-0924edc26ba2-kube-api-access-sgqfc\") pod \"node-resolver-62r4h\" (UID: \"3ba53ae5-7a20-4763-a164-0924edc26ba2\") " pod="openshift-dns/node-resolver-62r4h" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948310 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-socket-dir\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948345 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-multus-cni-dir\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948364 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-sysconfig\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948387 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-system-cni-dir\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948390 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-sysctl-d\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948434 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-cnibin\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948460 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948475 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-os-release\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948487 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwsjw\" (UniqueName: \"kubernetes.io/projected/e973bb49-d732-4f0d-b518-7f25d5ebd9ff-kube-api-access-dwsjw\") pod \"node-ca-rnz7t\" (UID: \"e973bb49-d732-4f0d-b518-7f25d5ebd9ff\") " pod="openshift-image-registry/node-ca-rnz7t" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948507 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/50aacd38-b488-40dc-98c2-cad52a033e3e-iptables-alerter-script\") pod \"iptables-alerter-282w9\" (UID: \"50aacd38-b488-40dc-98c2-cad52a033e3e\") " pod="openshift-network-operator/iptables-alerter-282w9" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948515 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/336c95f4-fe3d-4af2-b2bf-395b7cae9633-konnectivity-ca\") pod \"konnectivity-agent-nmqk9\" (UID: \"336c95f4-fe3d-4af2-b2bf-395b7cae9633\") " pod="kube-system/konnectivity-agent-nmqk9" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948511 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-cnibin\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.950235 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948563 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c1272af-f33d-4d09-9e09-bb846baccbc0-ovnkube-config\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.951048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948596 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c1272af-f33d-4d09-9e09-bb846baccbc0-ovn-node-metrics-cert\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.951048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948625 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbmdq\" (UniqueName: \"kubernetes.io/projected/4c1272af-f33d-4d09-9e09-bb846baccbc0-kube-api-access-nbmdq\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.951048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948670 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-kubernetes\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:48.951048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948696 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-os-release\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.951048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948720 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-etc-selinux\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.951048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948744 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ba53ae5-7a20-4763-a164-0924edc26ba2-hosts-file\") pod \"node-resolver-62r4h\" (UID: \"3ba53ae5-7a20-4763-a164-0924edc26ba2\") " pod="openshift-dns/node-resolver-62r4h" Jul 10 20:57:48.951048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948769 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbl5k\" (UniqueName: \"kubernetes.io/projected/ca5ea89b-dd36-4208-b892-a9b1176c0234-kube-api-access-vbl5k\") pod \"network-metrics-daemon-lk7sj\" (UID: \"ca5ea89b-dd36-4208-b892-a9b1176c0234\") " pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:48.951048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948794 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-cni-netd\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.951048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948815 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c1272af-f33d-4d09-9e09-bb846baccbc0-env-overrides\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:48.951048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948865 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2e37f80-e59b-4aae-a1ef-bc808092cd22-etc-selinux\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.951048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948771 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3fc26a94-4249-49e4-af9c-f4deea65bc50-os-release\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.951048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.948969 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3ba53ae5-7a20-4763-a164-0924edc26ba2-hosts-file\") pod \"node-resolver-62r4h\" (UID: \"3ba53ae5-7a20-4763-a164-0924edc26ba2\") " pod="openshift-dns/node-resolver-62r4h" Jul 10 20:57:48.951048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.949594 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.951708 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.951685 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 20:57:48.956053 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.955989 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhqnl\" (UniqueName: \"kubernetes.io/projected/50aacd38-b488-40dc-98c2-cad52a033e3e-kube-api-access-jhqnl\") pod \"iptables-alerter-282w9\" (UID: \"50aacd38-b488-40dc-98c2-cad52a033e3e\") " pod="openshift-network-operator/iptables-alerter-282w9" Jul 10 20:57:48.956572 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.956547 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bmh4\" (UniqueName: \"kubernetes.io/projected/3fc26a94-4249-49e4-af9c-f4deea65bc50-kube-api-access-8bmh4\") pod \"multus-6kpsn\" (UID: \"3fc26a94-4249-49e4-af9c-f4deea65bc50\") " pod="openshift-multus/multus-6kpsn" Jul 10 20:57:48.956677 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.956550 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zflc8\" (UniqueName: \"kubernetes.io/projected/e2e37f80-e59b-4aae-a1ef-bc808092cd22-kube-api-access-zflc8\") pod \"aws-ebs-csi-driver-node-85gbf\" (UID: \"e2e37f80-e59b-4aae-a1ef-bc808092cd22\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:48.956677 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.956551 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngsv4\" (UniqueName: \"kubernetes.io/projected/caad5f13-3ab6-4db2-bfbe-c5e9781d5f76-kube-api-access-ngsv4\") pod \"multus-additional-cni-plugins-kv7p8\" (UID: \"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76\") " pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:48.957968 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.957883 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwsjw\" (UniqueName: \"kubernetes.io/projected/e973bb49-d732-4f0d-b518-7f25d5ebd9ff-kube-api-access-dwsjw\") pod \"node-ca-rnz7t\" (UID: \"e973bb49-d732-4f0d-b518-7f25d5ebd9ff\") " pod="openshift-image-registry/node-ca-rnz7t" Jul 10 20:57:48.962438 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:48.961744 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgqfc\" (UniqueName: \"kubernetes.io/projected/3ba53ae5-7a20-4763-a164-0924edc26ba2-kube-api-access-sgqfc\") pod \"node-resolver-62r4h\" (UID: \"3ba53ae5-7a20-4763-a164-0924edc26ba2\") " pod="openshift-dns/node-resolver-62r4h" Jul 10 20:57:49.049133 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049103 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-systemd-units\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049133 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049141 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/336c95f4-fe3d-4af2-b2bf-395b7cae9633-agent-certs\") pod \"konnectivity-agent-nmqk9\" (UID: \"336c95f4-fe3d-4af2-b2bf-395b7cae9633\") " pod="kube-system/konnectivity-agent-nmqk9" Jul 10 20:57:49.049369 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049165 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-cni-bin\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049369 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049186 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-modprobe-d\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.049369 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049210 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-run-netns\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049369 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049229 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-systemd-units\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049369 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049297 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-cni-bin\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049369 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049244 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-run-systemd\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049369 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049336 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-run-openvswitch\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049369 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049367 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-modprobe-d\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.049784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049377 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-run-systemd\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049420 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-run-ovn-kubernetes\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049429 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-run-netns\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049475 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-run-openvswitch\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049508 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049539 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049596 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-run-ovn-kubernetes\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049640 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-var-lib-openvswitch\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049665 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-lib-modules\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.049784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049718 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-var-lib-openvswitch\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.049784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049763 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs\") pod \"network-metrics-daemon-lk7sj\" (UID: \"ca5ea89b-dd36-4208-b892-a9b1176c0234\") " pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:49.049784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049789 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-dbus\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049814 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-node-log\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049835 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-lib-modules\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049844 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-sysconfig\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049869 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-sysctl-d\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049895 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/336c95f4-fe3d-4af2-b2bf-395b7cae9633-konnectivity-ca\") pod \"konnectivity-agent-nmqk9\" (UID: \"336c95f4-fe3d-4af2-b2bf-395b7cae9633\") " pod="kube-system/konnectivity-agent-nmqk9" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049918 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c1272af-f33d-4d09-9e09-bb846baccbc0-ovnkube-config\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049927 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-dbus\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049939 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c1272af-f33d-4d09-9e09-bb846baccbc0-ovn-node-metrics-cert\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049954 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-sysconfig\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbmdq\" (UniqueName: \"kubernetes.io/projected/4c1272af-f33d-4d09-9e09-bb846baccbc0-kube-api-access-nbmdq\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049967 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-node-log\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.049897 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.049998 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-kubernetes\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050028 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbl5k\" (UniqueName: \"kubernetes.io/projected/ca5ea89b-dd36-4208-b892-a9b1176c0234-kube-api-access-vbl5k\") pod \"network-metrics-daemon-lk7sj\" (UID: \"ca5ea89b-dd36-4208-b892-a9b1176c0234\") " pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050052 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-cni-netd\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050077 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c1272af-f33d-4d09-9e09-bb846baccbc0-env-overrides\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050102 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-var-lib-kubelet\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.050318 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050124 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-host\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050152 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-slash\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050175 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ec81adf8-e539-414a-9bde-d607f042b620-etc-tuned\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050187 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-cni-netd\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050199 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-kubelet-config\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050223 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-systemd\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050238 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-kubernetes\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050305 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050336 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-run-ovn\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050360 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-log-socket\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050386 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c1272af-f33d-4d09-9e09-bb846baccbc0-ovnkube-script-lib\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050423 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-sysctl-conf\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050448 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-kubelet\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050581 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/336c95f4-fe3d-4af2-b2bf-395b7cae9633-konnectivity-ca\") pod \"konnectivity-agent-nmqk9\" (UID: \"336c95f4-fe3d-4af2-b2bf-395b7cae9633\") " pod="kube-system/konnectivity-agent-nmqk9" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050624 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-sysctl-conf\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050654 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-etc-openvswitch\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.050683 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs podName:ca5ea89b-dd36-4208-b892-a9b1176c0234 nodeName:}" failed. No retries permitted until 2026-07-10 20:57:49.550660294 +0000 UTC m=+3.134355893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs") pod "network-metrics-daemon-lk7sj" (UID: "ca5ea89b-dd36-4208-b892-a9b1176c0234") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:57:49.051196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050690 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-etc-openvswitch\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050723 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-kubelet\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-run\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050769 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec81adf8-e539-414a-9bde-d607f042b620-tmp\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050785 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-sysctl-d\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050798 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcjp8\" (UniqueName: \"kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8\") pod \"network-check-target-8lb77\" (UID: \"03703dd3-bccf-4be3-bd9d-68c093249f6c\") " pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050823 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-sys\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050855 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rfnn\" (UniqueName: \"kubernetes.io/projected/ec81adf8-e539-414a-9bde-d607f042b620-kube-api-access-9rfnn\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.050936 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-etc-systemd\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.051012 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-kubelet-config\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.051059 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-run-ovn\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.051102 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-var-lib-kubelet\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.051315 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-host\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.051369 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c1272af-f33d-4d09-9e09-bb846baccbc0-env-overrides\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.051435 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-log-socket\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.051585 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-sys\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.051588 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ec81adf8-e539-414a-9bde-d607f042b620-run\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.051745 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c1272af-f33d-4d09-9e09-bb846baccbc0-host-slash\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.052809 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.051782 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:57:49.053725 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.051782 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c1272af-f33d-4d09-9e09-bb846baccbc0-ovnkube-config\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.053725 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.051850 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret podName:94f8ad00-553c-47e0-8453-cdbe5b2a2b39 nodeName:}" failed. No retries permitted until 2026-07-10 20:57:49.551833121 +0000 UTC m=+3.135528705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret") pod "global-pull-secret-syncer-9788m" (UID: "94f8ad00-553c-47e0-8453-cdbe5b2a2b39") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:57:49.053725 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.052215 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c1272af-f33d-4d09-9e09-bb846baccbc0-ovnkube-script-lib\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.053725 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.052232 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/336c95f4-fe3d-4af2-b2bf-395b7cae9633-agent-certs\") pod \"konnectivity-agent-nmqk9\" (UID: \"336c95f4-fe3d-4af2-b2bf-395b7cae9633\") " pod="kube-system/konnectivity-agent-nmqk9" Jul 10 20:57:49.053725 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.052896 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ec81adf8-e539-414a-9bde-d607f042b620-etc-tuned\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.053725 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.053389 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c1272af-f33d-4d09-9e09-bb846baccbc0-ovn-node-metrics-cert\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.054021 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.054001 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec81adf8-e539-414a-9bde-d607f042b620-tmp\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.059876 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.059773 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:57:49.059876 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.059795 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:57:49.059876 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.059808 2566 projected.go:196] Error preparing data for projected volume kube-api-access-bcjp8 for pod openshift-network-diagnostics/network-check-target-8lb77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:57:49.060105 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.059912 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8 podName:03703dd3-bccf-4be3-bd9d-68c093249f6c nodeName:}" failed. No retries permitted until 2026-07-10 20:57:49.559893583 +0000 UTC m=+3.143589164 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bcjp8" (UniqueName: "kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8") pod "network-check-target-8lb77" (UID: "03703dd3-bccf-4be3-bd9d-68c093249f6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:57:49.062301 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.062278 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbl5k\" (UniqueName: \"kubernetes.io/projected/ca5ea89b-dd36-4208-b892-a9b1176c0234-kube-api-access-vbl5k\") pod \"network-metrics-daemon-lk7sj\" (UID: \"ca5ea89b-dd36-4208-b892-a9b1176c0234\") " pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:49.063283 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.063259 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rfnn\" (UniqueName: \"kubernetes.io/projected/ec81adf8-e539-414a-9bde-d607f042b620-kube-api-access-9rfnn\") pod \"tuned-t5v8m\" (UID: \"ec81adf8-e539-414a-9bde-d607f042b620\") " pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.063470 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.063446 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbmdq\" (UniqueName: \"kubernetes.io/projected/4c1272af-f33d-4d09-9e09-bb846baccbc0-kube-api-access-nbmdq\") pod \"ovnkube-node-j5q89\" (UID: \"4c1272af-f33d-4d09-9e09-bb846baccbc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.145651 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.145571 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kv7p8" Jul 10 20:57:49.156625 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.156603 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-282w9" Jul 10 20:57:49.169344 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.169321 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" Jul 10 20:57:49.175110 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.175090 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-62r4h" Jul 10 20:57:49.183750 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.183729 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rnz7t" Jul 10 20:57:49.195263 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.195247 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6kpsn" Jul 10 20:57:49.199961 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.199944 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nmqk9" Jul 10 20:57:49.209511 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.209495 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:57:49.214076 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.214060 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" Jul 10 20:57:49.554675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.554596 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:49.554675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.554662 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs\") pod \"network-metrics-daemon-lk7sj\" (UID: \"ca5ea89b-dd36-4208-b892-a9b1176c0234\") " pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:49.554871 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.554742 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:57:49.554871 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.554782 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:57:49.554871 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.554801 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret podName:94f8ad00-553c-47e0-8453-cdbe5b2a2b39 nodeName:}" failed. No retries permitted until 2026-07-10 20:57:50.554785021 +0000 UTC m=+4.138480620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret") pod "global-pull-secret-syncer-9788m" (UID: "94f8ad00-553c-47e0-8453-cdbe5b2a2b39") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:57:49.554871 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.554848 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs podName:ca5ea89b-dd36-4208-b892-a9b1176c0234 nodeName:}" failed. No retries permitted until 2026-07-10 20:57:50.554830488 +0000 UTC m=+4.138526075 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs") pod "network-metrics-daemon-lk7sj" (UID: "ca5ea89b-dd36-4208-b892-a9b1176c0234") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:57:49.655921 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.655881 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcjp8\" (UniqueName: \"kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8\") pod \"network-check-target-8lb77\" (UID: \"03703dd3-bccf-4be3-bd9d-68c093249f6c\") " pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:57:49.656100 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.656025 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:57:49.656100 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.656045 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:57:49.656100 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.656055 2566 projected.go:196] Error preparing data for projected volume kube-api-access-bcjp8 for pod openshift-network-diagnostics/network-check-target-8lb77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:57:49.656260 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:49.656113 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8 podName:03703dd3-bccf-4be3-bd9d-68c093249f6c nodeName:}" failed. No retries permitted until 2026-07-10 20:57:50.656098006 +0000 UTC m=+4.239793587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bcjp8" (UniqueName: "kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8") pod "network-check-target-8lb77" (UID: "03703dd3-bccf-4be3-bd9d-68c093249f6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:57:49.881142 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.881064 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 20:52:48 +0000 UTC" deadline="2028-04-21 04:04:52.067303374 +0000 UTC" Jul 10 20:57:49.881142 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:49.881093 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15607h7m2.186214452s" Jul 10 20:57:49.980237 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:49.980205 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode973bb49_d732_4f0d_b518_7f25d5ebd9ff.slice/crio-81b617e635f22af02aca2a5eea578f00cda5ace12b2fcebc0730b3b632e16b7f WatchSource:0}: Error finding container 81b617e635f22af02aca2a5eea578f00cda5ace12b2fcebc0730b3b632e16b7f: Status 404 returned error can't find the container with id 81b617e635f22af02aca2a5eea578f00cda5ace12b2fcebc0730b3b632e16b7f Jul 10 20:57:49.988504 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:49.988247 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336c95f4_fe3d_4af2_b2bf_395b7cae9633.slice/crio-4467e66135035ce6a7c975f551a0a08c731b4e7cae23c478eb2517ab1a9d5c25 WatchSource:0}: Error finding container 4467e66135035ce6a7c975f551a0a08c731b4e7cae23c478eb2517ab1a9d5c25: Status 404 returned error can't find the container with id 4467e66135035ce6a7c975f551a0a08c731b4e7cae23c478eb2517ab1a9d5c25 Jul 10 20:57:49.989049 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:49.989026 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba53ae5_7a20_4763_a164_0924edc26ba2.slice/crio-c7e67229d5621bfeae641950c497c15b9043784a8889904623e5ff90fd3be61c WatchSource:0}: Error finding container c7e67229d5621bfeae641950c497c15b9043784a8889904623e5ff90fd3be61c: Status 404 returned error can't find the container with id c7e67229d5621bfeae641950c497c15b9043784a8889904623e5ff90fd3be61c Jul 10 20:57:49.990219 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:49.990199 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50aacd38_b488_40dc_98c2_cad52a033e3e.slice/crio-09989906ce38af5cbc2539ee0246468819eb94261bc452173d14d27636c186c1 WatchSource:0}: Error finding container 09989906ce38af5cbc2539ee0246468819eb94261bc452173d14d27636c186c1: Status 404 returned error can't find the container with id 09989906ce38af5cbc2539ee0246468819eb94261bc452173d14d27636c186c1 Jul 10 20:57:49.991601 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:49.991582 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c1272af_f33d_4d09_9e09_bb846baccbc0.slice/crio-34594c533e2fdf80fcccc5421338ea04e8f1a12ce4c7ec32e2cefd4dd6e33a7a WatchSource:0}: Error finding container 34594c533e2fdf80fcccc5421338ea04e8f1a12ce4c7ec32e2cefd4dd6e33a7a: Status 404 returned error can't find the container with id 34594c533e2fdf80fcccc5421338ea04e8f1a12ce4c7ec32e2cefd4dd6e33a7a Jul 10 20:57:49.992546 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:49.992522 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaad5f13_3ab6_4db2_bfbe_c5e9781d5f76.slice/crio-409884c6dedb63d5b26afc9bc6db4bcc6a058f1f76d4aba35a8ba100345c89a2 WatchSource:0}: Error finding container 409884c6dedb63d5b26afc9bc6db4bcc6a058f1f76d4aba35a8ba100345c89a2: Status 404 returned error can't find the container with id 409884c6dedb63d5b26afc9bc6db4bcc6a058f1f76d4aba35a8ba100345c89a2 Jul 10 20:57:49.993816 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:49.993765 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fc26a94_4249_49e4_af9c_f4deea65bc50.slice/crio-741f826becc9abb50d16096c050b495e63683f93b5bbb1fc394fdf1aea1b8850 WatchSource:0}: Error finding container 741f826becc9abb50d16096c050b495e63683f93b5bbb1fc394fdf1aea1b8850: Status 404 returned error can't find the container with id 741f826becc9abb50d16096c050b495e63683f93b5bbb1fc394fdf1aea1b8850 Jul 10 20:57:49.994742 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:49.994714 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec81adf8_e539_414a_9bde_d607f042b620.slice/crio-6579ee36294b4706143113f3e08c69251c22e617c65952d394e81ab64cfc9a95 WatchSource:0}: Error finding container 6579ee36294b4706143113f3e08c69251c22e617c65952d394e81ab64cfc9a95: Status 404 returned error can't find the container with id 6579ee36294b4706143113f3e08c69251c22e617c65952d394e81ab64cfc9a95 Jul 10 20:57:49.996646 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:57:49.995912 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e37f80_e59b_4aae_a1ef_bc808092cd22.slice/crio-87c69431a3c79699415c2282129faf454c5eeb380a355108bc85d9ab2ac071dd WatchSource:0}: Error finding container 87c69431a3c79699415c2282129faf454c5eeb380a355108bc85d9ab2ac071dd: Status 404 returned error can't find the container with id 87c69431a3c79699415c2282129faf454c5eeb380a355108bc85d9ab2ac071dd Jul 10 20:57:50.062003 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.061980 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:50.062112 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:50.062086 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:57:50.072767 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.072736 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kv7p8" event={"ID":"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76","Type":"ContainerStarted","Data":"409884c6dedb63d5b26afc9bc6db4bcc6a058f1f76d4aba35a8ba100345c89a2"} Jul 10 20:57:50.074077 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.074052 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" event={"ID":"4c1272af-f33d-4d09-9e09-bb846baccbc0","Type":"ContainerStarted","Data":"34594c533e2fdf80fcccc5421338ea04e8f1a12ce4c7ec32e2cefd4dd6e33a7a"} Jul 10 20:57:50.075316 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.075295 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rnz7t" event={"ID":"e973bb49-d732-4f0d-b518-7f25d5ebd9ff","Type":"ContainerStarted","Data":"81b617e635f22af02aca2a5eea578f00cda5ace12b2fcebc0730b3b632e16b7f"} Jul 10 20:57:50.076538 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.076518 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6kpsn" event={"ID":"3fc26a94-4249-49e4-af9c-f4deea65bc50","Type":"ContainerStarted","Data":"741f826becc9abb50d16096c050b495e63683f93b5bbb1fc394fdf1aea1b8850"} Jul 10 20:57:50.077766 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.077746 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-282w9" event={"ID":"50aacd38-b488-40dc-98c2-cad52a033e3e","Type":"ContainerStarted","Data":"09989906ce38af5cbc2539ee0246468819eb94261bc452173d14d27636c186c1"} Jul 10 20:57:50.078900 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.078881 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-62r4h" event={"ID":"3ba53ae5-7a20-4763-a164-0924edc26ba2","Type":"ContainerStarted","Data":"c7e67229d5621bfeae641950c497c15b9043784a8889904623e5ff90fd3be61c"} Jul 10 20:57:50.080037 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.080010 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nmqk9" event={"ID":"336c95f4-fe3d-4af2-b2bf-395b7cae9633","Type":"ContainerStarted","Data":"4467e66135035ce6a7c975f551a0a08c731b4e7cae23c478eb2517ab1a9d5c25"} Jul 10 20:57:50.081304 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.081284 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" event={"ID":"e2e37f80-e59b-4aae-a1ef-bc808092cd22","Type":"ContainerStarted","Data":"87c69431a3c79699415c2282129faf454c5eeb380a355108bc85d9ab2ac071dd"} Jul 10 20:57:50.082303 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.082279 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" event={"ID":"ec81adf8-e539-414a-9bde-d607f042b620","Type":"ContainerStarted","Data":"6579ee36294b4706143113f3e08c69251c22e617c65952d394e81ab64cfc9a95"} Jul 10 20:57:50.562664 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.561920 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:50.562664 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.562009 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs\") pod \"network-metrics-daemon-lk7sj\" (UID: \"ca5ea89b-dd36-4208-b892-a9b1176c0234\") " pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:50.562664 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:50.562135 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:57:50.562664 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:50.562198 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs podName:ca5ea89b-dd36-4208-b892-a9b1176c0234 nodeName:}" failed. No retries permitted until 2026-07-10 20:57:52.562179395 +0000 UTC m=+6.145874979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs") pod "network-metrics-daemon-lk7sj" (UID: "ca5ea89b-dd36-4208-b892-a9b1176c0234") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:57:50.562664 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:50.562267 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:57:50.562664 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:50.562302 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret podName:94f8ad00-553c-47e0-8453-cdbe5b2a2b39 nodeName:}" failed. No retries permitted until 2026-07-10 20:57:52.562291094 +0000 UTC m=+6.145986679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret") pod "global-pull-secret-syncer-9788m" (UID: "94f8ad00-553c-47e0-8453-cdbe5b2a2b39") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:57:50.662917 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.662881 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcjp8\" (UniqueName: \"kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8\") pod \"network-check-target-8lb77\" (UID: \"03703dd3-bccf-4be3-bd9d-68c093249f6c\") " pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:57:50.663146 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:50.663119 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:57:50.663146 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:50.663141 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:57:50.663278 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:50.663155 2566 projected.go:196] Error preparing data for projected volume kube-api-access-bcjp8 for pod openshift-network-diagnostics/network-check-target-8lb77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:57:50.663278 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:50.663220 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8 podName:03703dd3-bccf-4be3-bd9d-68c093249f6c nodeName:}" failed. No retries permitted until 2026-07-10 20:57:52.663200264 +0000 UTC m=+6.246895848 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bcjp8" (UniqueName: "kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8") pod "network-check-target-8lb77" (UID: "03703dd3-bccf-4be3-bd9d-68c093249f6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:57:50.882061 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.881770 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 20:52:48 +0000 UTC" deadline="2028-03-24 15:06:12.147697968 +0000 UTC" Jul 10 20:57:50.882061 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:50.881836 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14946h8m21.2658666s" Jul 10 20:57:51.064682 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:51.064654 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:57:51.064837 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:51.064777 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:57:51.065357 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:51.065189 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:51.065357 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:51.065284 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:57:51.106969 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:51.106934 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" event={"ID":"1cad2d201741314c351e55ceddd77eeb","Type":"ContainerStarted","Data":"c41e1b113a4aac82240a0279ca710498e5e1bfd4da45cff994494f93c18c7db5"} Jul 10 20:57:51.117148 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:51.117116 2566 generic.go:356] "Generic (PLEG): container finished" podID="5275d30bb107dec4f3d86b0471c4d693" containerID="9a6248585d8214e2c2f996cd2f8d7da02cd2c3465116ad17fe50c2aa675cd673" exitCode=0 Jul 10 20:57:51.117311 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:51.117162 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" event={"ID":"5275d30bb107dec4f3d86b0471c4d693","Type":"ContainerDied","Data":"9a6248585d8214e2c2f996cd2f8d7da02cd2c3465116ad17fe50c2aa675cd673"} Jul 10 20:57:51.137999 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:51.137951 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-227.ec2.internal" podStartSLOduration=3.137931801 podStartE2EDuration="3.137931801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:57:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:57:51.121559347 +0000 UTC m=+4.705254950" watchObservedRunningTime="2026-07-10 20:57:51.137931801 +0000 UTC m=+4.721627399" Jul 10 20:57:52.062057 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:52.062023 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:52.062569 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:52.062201 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:57:52.132631 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:52.132598 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" event={"ID":"5275d30bb107dec4f3d86b0471c4d693","Type":"ContainerStarted","Data":"d9f3ea02eb9f5ddd47aead5a1af48405948731a827688d48a5f180a6e23c2ca1"} Jul 10 20:57:52.579805 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:52.579006 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:52.579805 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:52.579086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs\") pod \"network-metrics-daemon-lk7sj\" (UID: \"ca5ea89b-dd36-4208-b892-a9b1176c0234\") " pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:52.579805 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:52.579232 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:57:52.579805 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:52.579296 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs podName:ca5ea89b-dd36-4208-b892-a9b1176c0234 nodeName:}" failed. No retries permitted until 2026-07-10 20:57:56.5792754 +0000 UTC m=+10.162971182 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs") pod "network-metrics-daemon-lk7sj" (UID: "ca5ea89b-dd36-4208-b892-a9b1176c0234") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:57:52.579805 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:52.579693 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:57:52.579805 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:52.579745 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret podName:94f8ad00-553c-47e0-8453-cdbe5b2a2b39 nodeName:}" failed. No retries permitted until 2026-07-10 20:57:56.579729662 +0000 UTC m=+10.163425247 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret") pod "global-pull-secret-syncer-9788m" (UID: "94f8ad00-553c-47e0-8453-cdbe5b2a2b39") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:57:52.680048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:52.680011 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcjp8\" (UniqueName: \"kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8\") pod \"network-check-target-8lb77\" (UID: \"03703dd3-bccf-4be3-bd9d-68c093249f6c\") " pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:57:52.680244 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:52.680174 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:57:52.680244 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:52.680196 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:57:52.680244 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:52.680208 2566 projected.go:196] Error preparing data for projected volume kube-api-access-bcjp8 for pod openshift-network-diagnostics/network-check-target-8lb77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:57:52.680427 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:52.680267 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8 podName:03703dd3-bccf-4be3-bd9d-68c093249f6c nodeName:}" failed. No retries permitted until 2026-07-10 20:57:56.68024913 +0000 UTC m=+10.263944724 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bcjp8" (UniqueName: "kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8") pod "network-check-target-8lb77" (UID: "03703dd3-bccf-4be3-bd9d-68c093249f6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:57:53.062787 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:53.062708 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:57:53.063238 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:53.062848 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:57:53.063238 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:53.062962 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:53.063238 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:53.063080 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:57:54.061839 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:54.061807 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:54.062004 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:54.061947 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:57:55.062859 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:55.062349 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:57:55.062859 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:55.062494 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:57:55.062859 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:55.062617 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:55.062859 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:55.062715 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:57:56.062144 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:56.062054 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:56.062453 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:56.062216 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:57:56.614964 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:56.614730 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:56.614964 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:56.614839 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs\") pod \"network-metrics-daemon-lk7sj\" (UID: \"ca5ea89b-dd36-4208-b892-a9b1176c0234\") " pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:56.614964 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:56.614909 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:57:56.615560 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:56.614983 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret podName:94f8ad00-553c-47e0-8453-cdbe5b2a2b39 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:04.614963502 +0000 UTC m=+18.198659088 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret") pod "global-pull-secret-syncer-9788m" (UID: "94f8ad00-553c-47e0-8453-cdbe5b2a2b39") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:57:56.615560 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:56.615041 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:57:56.615560 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:56.615097 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs podName:ca5ea89b-dd36-4208-b892-a9b1176c0234 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:04.615078921 +0000 UTC m=+18.198774501 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs") pod "network-metrics-daemon-lk7sj" (UID: "ca5ea89b-dd36-4208-b892-a9b1176c0234") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:57:56.715948 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:56.715902 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcjp8\" (UniqueName: \"kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8\") pod \"network-check-target-8lb77\" (UID: \"03703dd3-bccf-4be3-bd9d-68c093249f6c\") " pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:57:56.716122 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:56.716058 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:57:56.716122 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:56.716077 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:57:56.716122 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:56.716089 2566 projected.go:196] Error preparing data for projected volume kube-api-access-bcjp8 for pod openshift-network-diagnostics/network-check-target-8lb77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:57:56.716279 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:56.716144 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8 podName:03703dd3-bccf-4be3-bd9d-68c093249f6c nodeName:}" failed. No retries permitted until 2026-07-10 20:58:04.716125488 +0000 UTC m=+18.299821082 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bcjp8" (UniqueName: "kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8") pod "network-check-target-8lb77" (UID: "03703dd3-bccf-4be3-bd9d-68c093249f6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:57:57.063609 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:57.063439 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:57:57.063609 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:57.063556 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:57:57.063808 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:57.063647 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:57.063808 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:57.063763 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:57:58.062073 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:58.061988 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:57:58.062535 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:58.062118 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:57:59.062457 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:59.062422 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:57:59.062981 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:57:59.062466 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:57:59.062981 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:59.062566 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:57:59.062981 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:57:59.062687 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:58:00.061797 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:00.061763 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:00.061971 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:00.061899 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:58:01.062598 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:01.062564 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:01.063070 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:01.062693 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:58:01.063070 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:01.062774 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:01.063070 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:01.062893 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:58:02.061737 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:02.061702 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:02.061913 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:02.061841 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:58:03.062508 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:03.062462 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:03.062893 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:03.062481 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:03.062893 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:03.062597 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:58:03.062893 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:03.062646 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:58:04.062166 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:04.062129 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:04.062389 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:04.062271 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:58:04.674233 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:04.674196 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:04.674675 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:04.674255 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs\") pod \"network-metrics-daemon-lk7sj\" (UID: \"ca5ea89b-dd36-4208-b892-a9b1176c0234\") " pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:04.674675 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:04.674349 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:58:04.674675 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:04.674351 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:58:04.674675 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:04.674421 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs podName:ca5ea89b-dd36-4208-b892-a9b1176c0234 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:20.67438811 +0000 UTC m=+34.258083695 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs") pod "network-metrics-daemon-lk7sj" (UID: "ca5ea89b-dd36-4208-b892-a9b1176c0234") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:58:04.674675 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:04.674441 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret podName:94f8ad00-553c-47e0-8453-cdbe5b2a2b39 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:20.674430803 +0000 UTC m=+34.258126383 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret") pod "global-pull-secret-syncer-9788m" (UID: "94f8ad00-553c-47e0-8453-cdbe5b2a2b39") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:58:04.775332 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:04.775300 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcjp8\" (UniqueName: \"kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8\") pod \"network-check-target-8lb77\" (UID: \"03703dd3-bccf-4be3-bd9d-68c093249f6c\") " pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:04.775502 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:04.775481 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:58:04.775558 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:04.775512 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:58:04.775558 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:04.775529 2566 projected.go:196] Error preparing data for projected volume kube-api-access-bcjp8 for pod openshift-network-diagnostics/network-check-target-8lb77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:58:04.775644 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:04.775594 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8 podName:03703dd3-bccf-4be3-bd9d-68c093249f6c nodeName:}" failed. No retries permitted until 2026-07-10 20:58:20.775576385 +0000 UTC m=+34.359271969 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bcjp8" (UniqueName: "kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8") pod "network-check-target-8lb77" (UID: "03703dd3-bccf-4be3-bd9d-68c093249f6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:58:05.062213 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:05.062135 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:05.062375 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:05.062146 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:05.062375 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:05.062279 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:58:05.062375 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:05.062343 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:58:06.062700 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:06.062668 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:06.063105 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:06.062783 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:58:07.062876 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:07.062841 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:07.063334 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:07.062934 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:58:07.063334 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:07.063017 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:07.063334 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:07.063102 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:58:08.061847 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:08.061808 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:08.062025 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:08.061938 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:58:09.062257 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.062068 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:09.063077 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.062114 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:09.063077 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:09.062349 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:58:09.063077 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:09.062410 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:58:09.171155 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.171112 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" event={"ID":"ec81adf8-e539-414a-9bde-d607f042b620","Type":"ContainerStarted","Data":"d6620a489b48e4823f04f218c4e845d85e8f2b77c5e818d92f03b0869e2b3f03"} Jul 10 20:58:09.172785 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.172753 2566 generic.go:356] "Generic (PLEG): container finished" podID="caad5f13-3ab6-4db2-bfbe-c5e9781d5f76" containerID="3916be4ee52370a0482bc19598b5719aa5337cf1e5a1d18b4666dc777a68db3e" exitCode=0 Jul 10 20:58:09.172920 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.172814 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kv7p8" event={"ID":"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76","Type":"ContainerDied","Data":"3916be4ee52370a0482bc19598b5719aa5337cf1e5a1d18b4666dc777a68db3e"} Jul 10 20:58:09.176543 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.176515 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" event={"ID":"4c1272af-f33d-4d09-9e09-bb846baccbc0","Type":"ContainerStarted","Data":"d2211881f3fef7a2062133eb543e09ef4eb078a3ea5ade584531ea59e1d02eb2"} Jul 10 20:58:09.176713 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.176574 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" event={"ID":"4c1272af-f33d-4d09-9e09-bb846baccbc0","Type":"ContainerStarted","Data":"03b66b42d472a8970a827e57036f8e4a7ae16b1413625e1bcb27cea7610723ee"} Jul 10 20:58:09.176713 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.176620 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" event={"ID":"4c1272af-f33d-4d09-9e09-bb846baccbc0","Type":"ContainerStarted","Data":"db52d8c8c07b5b99292915a4df6fabf73550939fe308d2350e03f63fcba480af"} Jul 10 20:58:09.176713 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.176666 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" event={"ID":"4c1272af-f33d-4d09-9e09-bb846baccbc0","Type":"ContainerStarted","Data":"579d580006cd8e92ac5d45423a3a7773becb801f37a9a148e6b4ad73a1b0d2f9"} Jul 10 20:58:09.176865 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.176714 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" event={"ID":"4c1272af-f33d-4d09-9e09-bb846baccbc0","Type":"ContainerStarted","Data":"d3f63cd47df54da57875a6851aa1d99a259d5091dbb1c68ef054329deb15f571"} Jul 10 20:58:09.176865 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.176764 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" event={"ID":"4c1272af-f33d-4d09-9e09-bb846baccbc0","Type":"ContainerStarted","Data":"6b1a64c2f716d1bebf0045aa2bbdc9f1115304859fdf101459c9af4a330ac6fd"} Jul 10 20:58:09.178185 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.178159 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rnz7t" event={"ID":"e973bb49-d732-4f0d-b518-7f25d5ebd9ff","Type":"ContainerStarted","Data":"18a2c27387ecd65f5bb4f5d5a0fa5bf28030c201a38d0617829638a4aefcc576"} Jul 10 20:58:09.179867 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.179842 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6kpsn" event={"ID":"3fc26a94-4249-49e4-af9c-f4deea65bc50","Type":"ContainerStarted","Data":"c6222b4222144f7b4b5ce9c85be1713eb11c2c6bc5f1684387052e59619c6a3a"} Jul 10 20:58:09.181290 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.181264 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-62r4h" event={"ID":"3ba53ae5-7a20-4763-a164-0924edc26ba2","Type":"ContainerStarted","Data":"406860023bb734cd63f86e2191973f2d0bd7fc513966ca0a235b478482a48fc1"} Jul 10 20:58:09.182853 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.182820 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nmqk9" event={"ID":"336c95f4-fe3d-4af2-b2bf-395b7cae9633","Type":"ContainerStarted","Data":"f4316c3f31c9373e53f01a63c632b395a20d229a4421e11d813fb4889b341895"} Jul 10 20:58:09.184439 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.184415 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" event={"ID":"e2e37f80-e59b-4aae-a1ef-bc808092cd22","Type":"ContainerStarted","Data":"bba01deb04d42dd765680f84e716a91618e999771816ad6cdb193664502134c6"} Jul 10 20:58:09.194117 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.194047 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-227.ec2.internal" podStartSLOduration=21.194034364 podStartE2EDuration="21.194034364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:57:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:57:52.150937827 +0000 UTC m=+5.734633429" watchObservedRunningTime="2026-07-10 20:58:09.194034364 +0000 UTC m=+22.777729966" Jul 10 20:58:09.210855 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.210811 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-t5v8m" podStartSLOduration=3.771183153 podStartE2EDuration="22.210798537s" totalImagesPullingTime="18.439615384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:09.194171964 +0000 UTC m=+22.777867559" watchObservedRunningTime="2026-07-10 20:58:09.210798537 +0000 UTC m=+22.794494263" Jul 10 20:58:09.211374 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.211320 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-62r4h" podStartSLOduration=3.763231768 podStartE2EDuration="22.211312056s" totalImagesPullingTime="18.448080288s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:09.210659998 +0000 UTC m=+22.794355600" watchObservedRunningTime="2026-07-10 20:58:09.211312056 +0000 UTC m=+22.795007660" Jul 10 20:58:09.231727 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.231695 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6kpsn" podStartSLOduration=3.738989361 podStartE2EDuration="22.231680061s" totalImagesPullingTime="18.4926907s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:09.231182268 +0000 UTC m=+22.814877871" watchObservedRunningTime="2026-07-10 20:58:09.231680061 +0000 UTC m=+22.815375663" Jul 10 20:58:09.275571 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.275514 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rnz7t" podStartSLOduration=3.820384181 podStartE2EDuration="22.275501178s" totalImagesPullingTime="18.455116997s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:09.275258131 +0000 UTC m=+22.858953735" watchObservedRunningTime="2026-07-10 20:58:09.275501178 +0000 UTC m=+22.859196780" Jul 10 20:58:09.293379 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:09.293332 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nmqk9" podStartSLOduration=11.837684095 podStartE2EDuration="22.293318717s" totalImagesPullingTime="10.455634622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:09.293260586 +0000 UTC m=+22.876956188" watchObservedRunningTime="2026-07-10 20:58:09.293318717 +0000 UTC m=+22.877014318" Jul 10 20:58:10.062674 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:10.062650 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:10.063066 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:10.062761 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:58:10.096004 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:10.095976 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 20:58:10.188196 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:10.188157 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-282w9" event={"ID":"50aacd38-b488-40dc-98c2-cad52a033e3e","Type":"ContainerStarted","Data":"335963fb5ce0084d59c5645bf5bdee4d3ed1df91e5c868a657732d13dad38bbe"} Jul 10 20:58:10.190190 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:10.190159 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" event={"ID":"e2e37f80-e59b-4aae-a1ef-bc808092cd22","Type":"ContainerStarted","Data":"57ac43b8ba2a3827e5e6261c2b88f51d26c09e2c5f7b7323f44ae8d7cb16390c"} Jul 10 20:58:10.204154 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:10.204109 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-282w9" podStartSLOduration=4.757264264 podStartE2EDuration="23.20409724s" totalImagesPullingTime="18.446832976s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:10.203956867 +0000 UTC m=+23.787652470" watchObservedRunningTime="2026-07-10 20:58:10.20409724 +0000 UTC m=+23.787792842" Jul 10 20:58:10.909856 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:10.909728 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T20:58:10.095999973Z","UUID":"89839220-7c57-41aa-99ef-2e25e1bd1f1d","Handler":null,"Name":"","Endpoint":""} Jul 10 20:58:10.911532 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:10.911498 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 20:58:10.911532 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:10.911534 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 20:58:11.062619 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:11.062575 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:11.062783 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:11.062705 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:58:11.063130 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:11.062803 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:11.063130 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:11.062924 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:58:12.062863 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:12.062647 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:12.063287 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:12.062991 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:58:12.198378 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:12.198330 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" event={"ID":"4c1272af-f33d-4d09-9e09-bb846baccbc0","Type":"ContainerStarted","Data":"571d246df97dc86302620b00aeb938bcc644ad144dc7159dd2c67c12a62c51ee"} Jul 10 20:58:12.200941 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:12.200910 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" event={"ID":"e2e37f80-e59b-4aae-a1ef-bc808092cd22","Type":"ContainerStarted","Data":"a1e9cdeac993ea10f8119becc5cd2bb314e92fe057da17e56a388a39ed26036a"} Jul 10 20:58:12.968595 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:12.968562 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nmqk9" Jul 10 20:58:12.969274 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:12.969248 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nmqk9" Jul 10 20:58:12.986181 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:12.986130 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85gbf" podStartSLOduration=4.893643145 podStartE2EDuration="25.986110511s" totalImagesPullingTime="21.092467366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:57:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:12.226647961 +0000 UTC m=+25.810343568" watchObservedRunningTime="2026-07-10 20:58:12.986110511 +0000 UTC m=+26.569806115" Jul 10 20:58:13.062296 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:13.062254 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:13.062296 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:13.062272 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:13.062557 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:13.062378 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:58:13.062557 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:13.062460 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:58:13.203726 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:13.203637 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nmqk9" Jul 10 20:58:13.204355 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:13.204032 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nmqk9" Jul 10 20:58:14.062837 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:14.062658 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:14.063005 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:14.062921 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:58:14.207541 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:14.207509 2566 generic.go:356] "Generic (PLEG): container finished" podID="caad5f13-3ab6-4db2-bfbe-c5e9781d5f76" containerID="0f99dccee3d2facfd091416db917bd46e055476dbfdb4b5a20f4132d2df96d8a" exitCode=0 Jul 10 20:58:14.208300 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:14.207589 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kv7p8" event={"ID":"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76","Type":"ContainerDied","Data":"0f99dccee3d2facfd091416db917bd46e055476dbfdb4b5a20f4132d2df96d8a"} Jul 10 20:58:14.211504 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:14.211474 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" event={"ID":"4c1272af-f33d-4d09-9e09-bb846baccbc0","Type":"ContainerStarted","Data":"401f182226d4762178f2f26db95595314bb8ab7b3956e08c1796ac6194661ce6"} Jul 10 20:58:14.211707 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:14.211687 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:58:14.211801 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:14.211756 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:58:14.211864 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:14.211812 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:58:14.226556 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:14.226532 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:58:14.226672 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:14.226624 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:58:14.259179 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:14.259134 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" podStartSLOduration=8.756337349 podStartE2EDuration="27.259118349s" totalImagesPullingTime="18.502781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:14.258370881 +0000 UTC m=+27.842066496" watchObservedRunningTime="2026-07-10 20:58:14.259118349 +0000 UTC m=+27.842813951" Jul 10 20:58:15.061963 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:15.061925 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:15.062173 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:15.061925 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:15.062173 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:15.062061 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:58:15.062354 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:15.062303 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:58:15.421906 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:15.421727 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9788m"] Jul 10 20:58:15.422236 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:15.421986 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:15.422236 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:15.422090 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:58:15.424664 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:15.424643 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8lb77"] Jul 10 20:58:15.424751 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:15.424715 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:15.424788 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:15.424777 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:58:15.447719 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:15.447690 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lk7sj"] Jul 10 20:58:15.447819 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:15.447794 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:15.447885 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:15.447869 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:58:16.218028 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:16.217991 2566 generic.go:356] "Generic (PLEG): container finished" podID="caad5f13-3ab6-4db2-bfbe-c5e9781d5f76" containerID="6820d8d7b37390cd4196f493a5450aa7ea6b78458cb422288545e59c45a9c8c6" exitCode=0 Jul 10 20:58:16.218239 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:16.218072 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kv7p8" event={"ID":"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76","Type":"ContainerDied","Data":"6820d8d7b37390cd4196f493a5450aa7ea6b78458cb422288545e59c45a9c8c6"} Jul 10 20:58:17.062773 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:17.062741 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:17.063446 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:17.062892 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:17.063446 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:17.062920 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:58:17.063446 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:17.062984 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:58:17.063446 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:17.063034 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:17.063446 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:17.063092 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:58:18.224069 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:18.224034 2566 generic.go:356] "Generic (PLEG): container finished" podID="caad5f13-3ab6-4db2-bfbe-c5e9781d5f76" containerID="3ddc86cac5b4371dbd725ea342de1bf17d6926b2e528cb04c31ddec13ab024c1" exitCode=0 Jul 10 20:58:18.224777 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:18.224087 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kv7p8" event={"ID":"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76","Type":"ContainerDied","Data":"3ddc86cac5b4371dbd725ea342de1bf17d6926b2e528cb04c31ddec13ab024c1"} Jul 10 20:58:19.062499 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:19.062359 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:19.062671 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:19.062359 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:19.062671 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:19.062507 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:58:19.062671 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:19.062520 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:19.062827 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:19.062664 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:58:19.062827 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:19.062770 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:58:20.694551 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:20.694511 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs\") pod \"network-metrics-daemon-lk7sj\" (UID: \"ca5ea89b-dd36-4208-b892-a9b1176c0234\") " pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:20.695112 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:20.694577 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:20.695112 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:20.694676 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:58:20.695112 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:20.694705 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:58:20.695112 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:20.694760 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs podName:ca5ea89b-dd36-4208-b892-a9b1176c0234 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:52.694743108 +0000 UTC m=+66.278438688 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs") pod "network-metrics-daemon-lk7sj" (UID: "ca5ea89b-dd36-4208-b892-a9b1176c0234") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:58:20.695112 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:20.694780 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret podName:94f8ad00-553c-47e0-8453-cdbe5b2a2b39 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:52.694770581 +0000 UTC m=+66.278466161 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret") pod "global-pull-secret-syncer-9788m" (UID: "94f8ad00-553c-47e0-8453-cdbe5b2a2b39") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:58:20.795488 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:20.795453 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcjp8\" (UniqueName: \"kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8\") pod \"network-check-target-8lb77\" (UID: \"03703dd3-bccf-4be3-bd9d-68c093249f6c\") " pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:20.795681 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:20.795654 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:58:20.795681 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:20.795678 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:58:20.795808 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:20.795693 2566 projected.go:196] Error preparing data for projected volume kube-api-access-bcjp8 for pod openshift-network-diagnostics/network-check-target-8lb77: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:58:20.795808 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:20.795758 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8 podName:03703dd3-bccf-4be3-bd9d-68c093249f6c nodeName:}" failed. No retries permitted until 2026-07-10 20:58:52.795737761 +0000 UTC m=+66.379433354 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bcjp8" (UniqueName: "kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8") pod "network-check-target-8lb77" (UID: "03703dd3-bccf-4be3-bd9d-68c093249f6c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:58:21.062692 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.062658 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:21.062879 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.062658 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:21.062879 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.062661 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:21.062879 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:21.062814 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lk7sj" podUID="ca5ea89b-dd36-4208-b892-a9b1176c0234" Jul 10 20:58:21.063081 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:21.062878 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8lb77" podUID="03703dd3-bccf-4be3-bd9d-68c093249f6c" Jul 10 20:58:21.063081 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:21.062975 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9788m" podUID="94f8ad00-553c-47e0-8453-cdbe5b2a2b39" Jul 10 20:58:21.488903 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.488874 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-227.ec2.internal" event="NodeReady" Jul 10 20:58:21.489073 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.489031 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 20:58:21.544903 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.544858 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mpbf9"] Jul 10 20:58:21.581173 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.581140 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fd78g"] Jul 10 20:58:21.581338 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.581275 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:21.584331 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.584310 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 20:58:21.584760 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.584740 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 20:58:21.585843 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.585823 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 20:58:21.585948 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.585827 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 20:58:21.585948 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.585882 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r272c\"" Jul 10 20:58:21.600753 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.600721 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mpbf9"] Jul 10 20:58:21.600869 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.600776 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fd78g"] Jul 10 20:58:21.600869 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.600855 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:21.603890 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.603866 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 20:58:21.604004 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.603934 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 20:58:21.604103 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.604046 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8hlxh\"" Jul 10 20:58:21.649579 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.649545 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-84gxc"] Jul 10 20:58:21.672661 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.672633 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-84gxc"] Jul 10 20:58:21.672830 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.672780 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-84gxc" Jul 10 20:58:21.676439 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.676413 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 20:58:21.676749 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.676693 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tq7hl\"" Jul 10 20:58:21.676749 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.676722 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 20:58:21.676749 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.676738 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 20:58:21.703506 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.703478 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:21.703853 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.703518 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd7ffba8-211e-43b8-bfe6-92696ef7035b-tmp-dir\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:21.703853 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.703545 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6hrl\" (UniqueName: \"kubernetes.io/projected/cd7ffba8-211e-43b8-bfe6-92696ef7035b-kube-api-access-p6hrl\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:21.703853 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.703634 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:21.703853 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.703662 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqwjt\" (UniqueName: \"kubernetes.io/projected/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-api-access-vqwjt\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:21.703853 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.703696 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-crio-socket\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:21.703853 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.703717 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cd7ffba8-211e-43b8-bfe6-92696ef7035b-config-volume\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:21.703853 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.703752 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:21.703853 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.703783 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-data-volume\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:21.805170 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.805081 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-crio-socket\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:21.805170 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.805117 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cd7ffba8-211e-43b8-bfe6-92696ef7035b-config-volume\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:21.805374 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.805269 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:21.805374 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.805311 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69b2e978-68f0-4f4d-acf2-1cc9ef90cd28-cert\") pod \"ingress-canary-84gxc\" (UID: \"69b2e978-68f0-4f4d-acf2-1cc9ef90cd28\") " pod="openshift-ingress-canary/ingress-canary-84gxc" Jul 10 20:58:21.805374 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.805339 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-crio-socket\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:21.805536 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:21.805384 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:58:21.805536 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.805426 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-data-volume\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:21.805536 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.805461 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9htwv\" (UniqueName: \"kubernetes.io/projected/69b2e978-68f0-4f4d-acf2-1cc9ef90cd28-kube-api-access-9htwv\") pod \"ingress-canary-84gxc\" (UID: \"69b2e978-68f0-4f4d-acf2-1cc9ef90cd28\") " pod="openshift-ingress-canary/ingress-canary-84gxc" Jul 10 20:58:21.805536 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:21.805479 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm podName:ff0579cc-b6f0-4cea-9840-0850d7fe6c31 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:22.305456229 +0000 UTC m=+35.889151809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mpbf9" (UID: "ff0579cc-b6f0-4cea-9840-0850d7fe6c31") : configmap "kube-rbac-proxy" not found Jul 10 20:58:21.805735 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.805549 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:21.805735 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.805573 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd7ffba8-211e-43b8-bfe6-92696ef7035b-tmp-dir\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:21.805735 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.805599 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6hrl\" (UniqueName: \"kubernetes.io/projected/cd7ffba8-211e-43b8-bfe6-92696ef7035b-kube-api-access-p6hrl\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:21.805735 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.805652 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:21.805735 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.805680 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqwjt\" (UniqueName: \"kubernetes.io/projected/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-api-access-vqwjt\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:21.805978 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.805772 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cd7ffba8-211e-43b8-bfe6-92696ef7035b-config-volume\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:21.805978 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.805802 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-data-volume\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:21.805978 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:21.805900 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:58:21.805978 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:21.805950 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls podName:ff0579cc-b6f0-4cea-9840-0850d7fe6c31 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:22.305932676 +0000 UTC m=+35.889628274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mpbf9" (UID: "ff0579cc-b6f0-4cea-9840-0850d7fe6c31") : secret "insights-runtime-extractor-tls" not found Jul 10 20:58:21.805978 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:21.805975 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:58:21.806170 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:21.806017 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls podName:cd7ffba8-211e-43b8-bfe6-92696ef7035b nodeName:}" failed. No retries permitted until 2026-07-10 20:58:22.306005742 +0000 UTC m=+35.889701322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls") pod "dns-default-fd78g" (UID: "cd7ffba8-211e-43b8-bfe6-92696ef7035b") : secret "dns-default-metrics-tls" not found Jul 10 20:58:21.806170 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.806060 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd7ffba8-211e-43b8-bfe6-92696ef7035b-tmp-dir\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:21.821411 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.821269 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqwjt\" (UniqueName: \"kubernetes.io/projected/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-api-access-vqwjt\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:21.821541 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.821269 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6hrl\" (UniqueName: \"kubernetes.io/projected/cd7ffba8-211e-43b8-bfe6-92696ef7035b-kube-api-access-p6hrl\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:21.856988 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.856962 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-62r4h_3ba53ae5-7a20-4763-a164-0924edc26ba2/dns-node-resolver/0.log" Jul 10 20:58:21.906261 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.906234 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9htwv\" (UniqueName: \"kubernetes.io/projected/69b2e978-68f0-4f4d-acf2-1cc9ef90cd28-kube-api-access-9htwv\") pod \"ingress-canary-84gxc\" (UID: \"69b2e978-68f0-4f4d-acf2-1cc9ef90cd28\") " pod="openshift-ingress-canary/ingress-canary-84gxc" Jul 10 20:58:21.906468 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.906365 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69b2e978-68f0-4f4d-acf2-1cc9ef90cd28-cert\") pod \"ingress-canary-84gxc\" (UID: \"69b2e978-68f0-4f4d-acf2-1cc9ef90cd28\") " pod="openshift-ingress-canary/ingress-canary-84gxc" Jul 10 20:58:21.909915 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.909887 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69b2e978-68f0-4f4d-acf2-1cc9ef90cd28-cert\") pod \"ingress-canary-84gxc\" (UID: \"69b2e978-68f0-4f4d-acf2-1cc9ef90cd28\") " pod="openshift-ingress-canary/ingress-canary-84gxc" Jul 10 20:58:21.916531 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.916506 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9htwv\" (UniqueName: \"kubernetes.io/projected/69b2e978-68f0-4f4d-acf2-1cc9ef90cd28-kube-api-access-9htwv\") pod \"ingress-canary-84gxc\" (UID: \"69b2e978-68f0-4f4d-acf2-1cc9ef90cd28\") " pod="openshift-ingress-canary/ingress-canary-84gxc" Jul 10 20:58:21.982375 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:21.982334 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-84gxc" Jul 10 20:58:22.309109 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:22.309082 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:22.309253 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:22.309124 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:22.309253 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:22.309163 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:22.309253 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:22.309235 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:58:22.309253 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:22.309252 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:58:22.309450 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:22.309280 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:58:22.309450 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:22.309306 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm podName:ff0579cc-b6f0-4cea-9840-0850d7fe6c31 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:23.309291272 +0000 UTC m=+36.892986853 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mpbf9" (UID: "ff0579cc-b6f0-4cea-9840-0850d7fe6c31") : configmap "kube-rbac-proxy" not found Jul 10 20:58:22.309450 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:22.309323 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls podName:cd7ffba8-211e-43b8-bfe6-92696ef7035b nodeName:}" failed. No retries permitted until 2026-07-10 20:58:23.3093159 +0000 UTC m=+36.893011493 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls") pod "dns-default-fd78g" (UID: "cd7ffba8-211e-43b8-bfe6-92696ef7035b") : secret "dns-default-metrics-tls" not found Jul 10 20:58:22.309450 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:22.309336 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls podName:ff0579cc-b6f0-4cea-9840-0850d7fe6c31 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:23.309328972 +0000 UTC m=+36.893024552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mpbf9" (UID: "ff0579cc-b6f0-4cea-9840-0850d7fe6c31") : secret "insights-runtime-extractor-tls" not found Jul 10 20:58:22.330820 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:22.330797 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-84gxc"] Jul 10 20:58:22.429769 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:58:22.429695 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69b2e978_68f0_4f4d_acf2_1cc9ef90cd28.slice/crio-0907cc827577fbd0475d9362856f8f8c8b9c10b50396a6af0ebff1ae0902af09 WatchSource:0}: Error finding container 0907cc827577fbd0475d9362856f8f8c8b9c10b50396a6af0ebff1ae0902af09: Status 404 returned error can't find the container with id 0907cc827577fbd0475d9362856f8f8c8b9c10b50396a6af0ebff1ae0902af09 Jul 10 20:58:23.062438 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.062392 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:23.062922 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.062387 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:23.062922 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.062419 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:23.067103 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.067079 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 20:58:23.067244 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.067142 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 20:58:23.067244 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.067143 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 20:58:23.067244 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.067198 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4fk7p\"" Jul 10 20:58:23.067244 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.067142 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8kn2h\"" Jul 10 20:58:23.067485 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.067082 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 20:58:23.216435 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.216410 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rnz7t_e973bb49-d732-4f0d-b518-7f25d5ebd9ff/node-ca/0.log" Jul 10 20:58:23.238863 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.238830 2566 generic.go:356] "Generic (PLEG): container finished" podID="caad5f13-3ab6-4db2-bfbe-c5e9781d5f76" containerID="f28b6ee1ef1d1fc3bd67d84cd6307c998f2ca85d45437c3077b991477ef21605" exitCode=0 Jul 10 20:58:23.239020 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.238911 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kv7p8" event={"ID":"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76","Type":"ContainerDied","Data":"f28b6ee1ef1d1fc3bd67d84cd6307c998f2ca85d45437c3077b991477ef21605"} Jul 10 20:58:23.240857 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.240261 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-84gxc" event={"ID":"69b2e978-68f0-4f4d-acf2-1cc9ef90cd28","Type":"ContainerStarted","Data":"0907cc827577fbd0475d9362856f8f8c8b9c10b50396a6af0ebff1ae0902af09"} Jul 10 20:58:23.317012 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.316981 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:23.317161 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.317048 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:23.317161 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:23.317114 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:23.317247 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:23.317219 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:58:23.317291 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:23.317243 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:58:23.317291 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:23.317282 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm podName:ff0579cc-b6f0-4cea-9840-0850d7fe6c31 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:25.317264072 +0000 UTC m=+38.900959673 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mpbf9" (UID: "ff0579cc-b6f0-4cea-9840-0850d7fe6c31") : configmap "kube-rbac-proxy" not found Jul 10 20:58:23.317357 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:23.317302 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls podName:cd7ffba8-211e-43b8-bfe6-92696ef7035b nodeName:}" failed. No retries permitted until 2026-07-10 20:58:25.31729194 +0000 UTC m=+38.900987521 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls") pod "dns-default-fd78g" (UID: "cd7ffba8-211e-43b8-bfe6-92696ef7035b") : secret "dns-default-metrics-tls" not found Jul 10 20:58:23.317357 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:23.317347 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:58:23.317461 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:23.317392 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls podName:ff0579cc-b6f0-4cea-9840-0850d7fe6c31 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:25.317373204 +0000 UTC m=+38.901068801 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mpbf9" (UID: "ff0579cc-b6f0-4cea-9840-0850d7fe6c31") : secret "insights-runtime-extractor-tls" not found Jul 10 20:58:24.246290 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:24.246097 2566 generic.go:356] "Generic (PLEG): container finished" podID="caad5f13-3ab6-4db2-bfbe-c5e9781d5f76" containerID="9d4187a42f48cc06036e5623c108e2c232c0abe5d3136e196462b14b2206ec48" exitCode=0 Jul 10 20:58:24.246290 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:24.246176 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kv7p8" event={"ID":"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76","Type":"ContainerDied","Data":"9d4187a42f48cc06036e5623c108e2c232c0abe5d3136e196462b14b2206ec48"} Jul 10 20:58:25.251908 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:25.251818 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kv7p8" event={"ID":"caad5f13-3ab6-4db2-bfbe-c5e9781d5f76","Type":"ContainerStarted","Data":"897d957efef45b374f8a0ef8b7d0d5df601c41365ec1b137b424f67f6948fdcb"} Jul 10 20:58:25.253265 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:25.253242 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-84gxc" event={"ID":"69b2e978-68f0-4f4d-acf2-1cc9ef90cd28","Type":"ContainerStarted","Data":"86c4cac16b0dfbf78547d0bc839a36eead529f7092a516d27f353d1fa03303ac"} Jul 10 20:58:25.281533 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:25.281484 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kv7p8" podStartSLOduration=9.088513602999999 podStartE2EDuration="38.281468573s" totalImagesPullingTime="29.19295497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:57:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:25.280453983 +0000 UTC m=+38.864149586" watchObservedRunningTime="2026-07-10 20:58:25.281468573 +0000 UTC m=+38.865164174" Jul 10 20:58:25.300772 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:25.300719 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-84gxc" podStartSLOduration=2.180497206 podStartE2EDuration="4.300704754s" totalImagesPullingTime="2.120207548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:25.299744941 +0000 UTC m=+38.883440542" watchObservedRunningTime="2026-07-10 20:58:25.300704754 +0000 UTC m=+38.884400355" Jul 10 20:58:25.336558 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:25.336525 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:25.336700 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:25.336580 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:25.336700 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:25.336658 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:58:25.336773 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:25.336697 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:25.336773 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:25.336731 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm podName:ff0579cc-b6f0-4cea-9840-0850d7fe6c31 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:29.336712368 +0000 UTC m=+42.920407966 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mpbf9" (UID: "ff0579cc-b6f0-4cea-9840-0850d7fe6c31") : configmap "kube-rbac-proxy" not found Jul 10 20:58:25.336773 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:25.336670 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:58:25.336888 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:25.336795 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls podName:cd7ffba8-211e-43b8-bfe6-92696ef7035b nodeName:}" failed. No retries permitted until 2026-07-10 20:58:29.336774881 +0000 UTC m=+42.920470461 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls") pod "dns-default-fd78g" (UID: "cd7ffba8-211e-43b8-bfe6-92696ef7035b") : secret "dns-default-metrics-tls" not found Jul 10 20:58:25.336888 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:25.336830 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:58:25.336888 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:25.336861 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls podName:ff0579cc-b6f0-4cea-9840-0850d7fe6c31 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:29.336851694 +0000 UTC m=+42.920547276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mpbf9" (UID: "ff0579cc-b6f0-4cea-9840-0850d7fe6c31") : secret "insights-runtime-extractor-tls" not found Jul 10 20:58:29.369004 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:29.368964 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:29.369474 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:29.369013 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:29.369474 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:29.369053 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:29.369474 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:29.369133 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:58:29.369474 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:29.369175 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:58:29.369474 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:29.369188 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm podName:ff0579cc-b6f0-4cea-9840-0850d7fe6c31 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:37.369173904 +0000 UTC m=+50.952869483 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mpbf9" (UID: "ff0579cc-b6f0-4cea-9840-0850d7fe6c31") : configmap "kube-rbac-proxy" not found Jul 10 20:58:29.369474 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:29.369242 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls podName:ff0579cc-b6f0-4cea-9840-0850d7fe6c31 nodeName:}" failed. No retries permitted until 2026-07-10 20:58:37.369224525 +0000 UTC m=+50.952920105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mpbf9" (UID: "ff0579cc-b6f0-4cea-9840-0850d7fe6c31") : secret "insights-runtime-extractor-tls" not found Jul 10 20:58:29.372482 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:29.372455 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd7ffba8-211e-43b8-bfe6-92696ef7035b-metrics-tls\") pod \"dns-default-fd78g\" (UID: \"cd7ffba8-211e-43b8-bfe6-92696ef7035b\") " pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:29.410450 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:29.410389 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:29.571265 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:29.571159 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fd78g"] Jul 10 20:58:29.573757 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:58:29.573729 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd7ffba8_211e_43b8_bfe6_92696ef7035b.slice/crio-38b0cc16b021b32a154d9500a807fd243df39b4e2d70f5176ed9e737de1f9b3e WatchSource:0}: Error finding container 38b0cc16b021b32a154d9500a807fd243df39b4e2d70f5176ed9e737de1f9b3e: Status 404 returned error can't find the container with id 38b0cc16b021b32a154d9500a807fd243df39b4e2d70f5176ed9e737de1f9b3e Jul 10 20:58:30.265345 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:30.265303 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fd78g" event={"ID":"cd7ffba8-211e-43b8-bfe6-92696ef7035b","Type":"ContainerStarted","Data":"38b0cc16b021b32a154d9500a807fd243df39b4e2d70f5176ed9e737de1f9b3e"} Jul 10 20:58:32.271296 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:32.271255 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fd78g" event={"ID":"cd7ffba8-211e-43b8-bfe6-92696ef7035b","Type":"ContainerStarted","Data":"1da310c937edf70ad6be0b60d9aa54f4bdaed04ffdabd8f6b66f66a060c31160"} Jul 10 20:58:32.271816 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:32.271344 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fd78g" event={"ID":"cd7ffba8-211e-43b8-bfe6-92696ef7035b","Type":"ContainerStarted","Data":"88347ae89579e9d5e29308a758d9249a059f49e1741cf00c89bca2bb9b2288b4"} Jul 10 20:58:32.271816 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:32.271387 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:32.293344 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:32.293294 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fd78g" podStartSLOduration=9.440911896 podStartE2EDuration="11.293278503s" totalImagesPullingTime="1.852366607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:58:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:32.291994833 +0000 UTC m=+45.875690434" watchObservedRunningTime="2026-07-10 20:58:32.293278503 +0000 UTC m=+45.876974099" Jul 10 20:58:37.430182 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:37.430139 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:37.430596 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:37.430200 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:37.430724 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:37.430706 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:37.432516 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:37.432494 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff0579cc-b6f0-4cea-9840-0850d7fe6c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mpbf9\" (UID: \"ff0579cc-b6f0-4cea-9840-0850d7fe6c31\") " pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:37.494843 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:37.494810 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mpbf9" Jul 10 20:58:37.663048 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:37.663018 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mpbf9"] Jul 10 20:58:38.288007 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:38.287970 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mpbf9" event={"ID":"ff0579cc-b6f0-4cea-9840-0850d7fe6c31","Type":"ContainerStarted","Data":"221e39dc08f8fd48b6e8d862f264630393ab3a76271020664ae11d77d572b581"} Jul 10 20:58:42.276304 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:42.276275 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fd78g" Jul 10 20:58:43.302198 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:43.302160 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mpbf9" event={"ID":"ff0579cc-b6f0-4cea-9840-0850d7fe6c31","Type":"ContainerStarted","Data":"6cdfde59d3b1f278baf0bd7e6ec9c6a9e4ef950147e0db7707efeb29064b32d2"} Jul 10 20:58:44.307352 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.307310 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mpbf9" event={"ID":"ff0579cc-b6f0-4cea-9840-0850d7fe6c31","Type":"ContainerStarted","Data":"f618b17a979b6de7753770d4c475f86886e3435113d9e5eae708eaa9f87c4a44"} Jul 10 20:58:44.884714 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.884679 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775f448-7k2mq"] Jul 10 20:58:44.886548 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.886528 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775f448-7k2mq" Jul 10 20:58:44.891299 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.891260 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 10 20:58:44.891299 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.891278 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-kv72w\"" Jul 10 20:58:44.891513 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.891304 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 10 20:58:44.891513 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.891277 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 10 20:58:44.895083 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.894874 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 10 20:58:44.915679 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.915616 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775f448-7k2mq"] Jul 10 20:58:44.928913 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.928885 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp"] Jul 10 20:58:44.930820 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.930795 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc"] Jul 10 20:58:44.930992 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.930952 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" Jul 10 20:58:44.932681 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.932662 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:44.933945 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.933921 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 10 20:58:44.942478 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.942456 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 10 20:58:44.942604 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.942465 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 10 20:58:44.942604 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.942580 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 10 20:58:44.943146 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.943127 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 10 20:58:44.962370 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.962333 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc"] Jul 10 20:58:44.963366 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.963342 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp"] Jul 10 20:58:44.987599 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.987548 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45kbd\" (UniqueName: \"kubernetes.io/projected/6d5632d4-a66d-4ee5-b796-9a2117e4e2b6-kube-api-access-45kbd\") pod \"klusterlet-addon-workmgr-86c6ff9d79-rh4vp\" (UID: \"6d5632d4-a66d-4ee5-b796-9a2117e4e2b6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" Jul 10 20:58:44.987784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.987617 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6d5632d4-a66d-4ee5-b796-9a2117e4e2b6-klusterlet-config\") pod \"klusterlet-addon-workmgr-86c6ff9d79-rh4vp\" (UID: \"6d5632d4-a66d-4ee5-b796-9a2117e4e2b6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" Jul 10 20:58:44.987784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.987649 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/77262f90-b294-41df-b9ca-7b76965a84f6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:44.987784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.987673 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgjf4\" (UniqueName: \"kubernetes.io/projected/77262f90-b294-41df-b9ca-7b76965a84f6-kube-api-access-tgjf4\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:44.987784 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.987764 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d5632d4-a66d-4ee5-b796-9a2117e4e2b6-tmp\") pod \"klusterlet-addon-workmgr-86c6ff9d79-rh4vp\" (UID: \"6d5632d4-a66d-4ee5-b796-9a2117e4e2b6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" Jul 10 20:58:44.987997 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.987788 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrqgd\" (UniqueName: \"kubernetes.io/projected/8820d5ab-9032-40c5-801c-445420e8a285-kube-api-access-jrqgd\") pod \"managed-serviceaccount-addon-agent-7c775f448-7k2mq\" (UID: \"8820d5ab-9032-40c5-801c-445420e8a285\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775f448-7k2mq" Jul 10 20:58:44.987997 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.987809 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/77262f90-b294-41df-b9ca-7b76965a84f6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:44.987997 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.987828 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/77262f90-b294-41df-b9ca-7b76965a84f6-hub\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:44.987997 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.987847 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77262f90-b294-41df-b9ca-7b76965a84f6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:44.987997 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.987956 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8820d5ab-9032-40c5-801c-445420e8a285-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c775f448-7k2mq\" (UID: \"8820d5ab-9032-40c5-801c-445420e8a285\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775f448-7k2mq" Jul 10 20:58:44.988177 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:44.988018 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/77262f90-b294-41df-b9ca-7b76965a84f6-ca\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:45.088639 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.088612 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6d5632d4-a66d-4ee5-b796-9a2117e4e2b6-klusterlet-config\") pod \"klusterlet-addon-workmgr-86c6ff9d79-rh4vp\" (UID: \"6d5632d4-a66d-4ee5-b796-9a2117e4e2b6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" Jul 10 20:58:45.088805 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.088649 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/77262f90-b294-41df-b9ca-7b76965a84f6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:45.088805 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.088673 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgjf4\" (UniqueName: \"kubernetes.io/projected/77262f90-b294-41df-b9ca-7b76965a84f6-kube-api-access-tgjf4\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:45.088943 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.088850 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d5632d4-a66d-4ee5-b796-9a2117e4e2b6-tmp\") pod \"klusterlet-addon-workmgr-86c6ff9d79-rh4vp\" (UID: \"6d5632d4-a66d-4ee5-b796-9a2117e4e2b6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" Jul 10 20:58:45.088943 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.088887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrqgd\" (UniqueName: \"kubernetes.io/projected/8820d5ab-9032-40c5-801c-445420e8a285-kube-api-access-jrqgd\") pod \"managed-serviceaccount-addon-agent-7c775f448-7k2mq\" (UID: \"8820d5ab-9032-40c5-801c-445420e8a285\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775f448-7k2mq" Jul 10 20:58:45.088943 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.088921 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/77262f90-b294-41df-b9ca-7b76965a84f6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:45.089095 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.088949 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/77262f90-b294-41df-b9ca-7b76965a84f6-hub\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:45.089095 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.088974 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77262f90-b294-41df-b9ca-7b76965a84f6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:45.089095 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.089018 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8820d5ab-9032-40c5-801c-445420e8a285-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c775f448-7k2mq\" (UID: \"8820d5ab-9032-40c5-801c-445420e8a285\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775f448-7k2mq" Jul 10 20:58:45.089095 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.089065 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/77262f90-b294-41df-b9ca-7b76965a84f6-ca\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:45.089292 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.089095 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45kbd\" (UniqueName: \"kubernetes.io/projected/6d5632d4-a66d-4ee5-b796-9a2117e4e2b6-kube-api-access-45kbd\") pod \"klusterlet-addon-workmgr-86c6ff9d79-rh4vp\" (UID: \"6d5632d4-a66d-4ee5-b796-9a2117e4e2b6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" Jul 10 20:58:45.089292 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.089278 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d5632d4-a66d-4ee5-b796-9a2117e4e2b6-tmp\") pod \"klusterlet-addon-workmgr-86c6ff9d79-rh4vp\" (UID: \"6d5632d4-a66d-4ee5-b796-9a2117e4e2b6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" Jul 10 20:58:45.089802 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.089774 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/77262f90-b294-41df-b9ca-7b76965a84f6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:45.091880 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.091852 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6d5632d4-a66d-4ee5-b796-9a2117e4e2b6-klusterlet-config\") pod \"klusterlet-addon-workmgr-86c6ff9d79-rh4vp\" (UID: \"6d5632d4-a66d-4ee5-b796-9a2117e4e2b6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" Jul 10 20:58:45.092006 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.091981 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8820d5ab-9032-40c5-801c-445420e8a285-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c775f448-7k2mq\" (UID: \"8820d5ab-9032-40c5-801c-445420e8a285\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775f448-7k2mq" Jul 10 20:58:45.092297 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.092253 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77262f90-b294-41df-b9ca-7b76965a84f6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:45.092297 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.092277 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/77262f90-b294-41df-b9ca-7b76965a84f6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:45.092535 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.092519 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/77262f90-b294-41df-b9ca-7b76965a84f6-ca\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:45.092772 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.092752 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/77262f90-b294-41df-b9ca-7b76965a84f6-hub\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:45.136956 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.136875 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgjf4\" (UniqueName: \"kubernetes.io/projected/77262f90-b294-41df-b9ca-7b76965a84f6-kube-api-access-tgjf4\") pod \"cluster-proxy-proxy-agent-5bb6b95559-k5xlc\" (UID: \"77262f90-b294-41df-b9ca-7b76965a84f6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:45.142761 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.142695 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45kbd\" (UniqueName: \"kubernetes.io/projected/6d5632d4-a66d-4ee5-b796-9a2117e4e2b6-kube-api-access-45kbd\") pod \"klusterlet-addon-workmgr-86c6ff9d79-rh4vp\" (UID: \"6d5632d4-a66d-4ee5-b796-9a2117e4e2b6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" Jul 10 20:58:45.145683 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.145662 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrqgd\" (UniqueName: \"kubernetes.io/projected/8820d5ab-9032-40c5-801c-445420e8a285-kube-api-access-jrqgd\") pod \"managed-serviceaccount-addon-agent-7c775f448-7k2mq\" (UID: \"8820d5ab-9032-40c5-801c-445420e8a285\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775f448-7k2mq" Jul 10 20:58:45.205815 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.205782 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775f448-7k2mq" Jul 10 20:58:45.244044 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.244001 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" Jul 10 20:58:45.250761 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:45.250741 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" Jul 10 20:58:46.191147 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:46.191109 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775f448-7k2mq"] Jul 10 20:58:46.195178 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:58:46.195142 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8820d5ab_9032_40c5_801c_445420e8a285.slice/crio-2cd5aa0de3cb8703061dc42902bfbc6293c936ed31eaf7d7dc4957a0dae70bb7 WatchSource:0}: Error finding container 2cd5aa0de3cb8703061dc42902bfbc6293c936ed31eaf7d7dc4957a0dae70bb7: Status 404 returned error can't find the container with id 2cd5aa0de3cb8703061dc42902bfbc6293c936ed31eaf7d7dc4957a0dae70bb7 Jul 10 20:58:46.195314 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:46.195180 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc"] Jul 10 20:58:46.198132 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:58:46.198105 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77262f90_b294_41df_b9ca_7b76965a84f6.slice/crio-b94204186d14f16201a7a1b2bc3cc8b7a6208c02fdf9eec8e3913e0feb3ab03e WatchSource:0}: Error finding container b94204186d14f16201a7a1b2bc3cc8b7a6208c02fdf9eec8e3913e0feb3ab03e: Status 404 returned error can't find the container with id b94204186d14f16201a7a1b2bc3cc8b7a6208c02fdf9eec8e3913e0feb3ab03e Jul 10 20:58:46.225014 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:46.224981 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp"] Jul 10 20:58:46.227773 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:58:46.227737 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d5632d4_a66d_4ee5_b796_9a2117e4e2b6.slice/crio-a39d0e8fa4d644be2fc1dd4c77136566b5d8b8e60058672d7fc850f97213b3ed WatchSource:0}: Error finding container a39d0e8fa4d644be2fc1dd4c77136566b5d8b8e60058672d7fc850f97213b3ed: Status 404 returned error can't find the container with id a39d0e8fa4d644be2fc1dd4c77136566b5d8b8e60058672d7fc850f97213b3ed Jul 10 20:58:46.228642 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:46.228614 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j5q89" Jul 10 20:58:46.315031 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:46.314936 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mpbf9" event={"ID":"ff0579cc-b6f0-4cea-9840-0850d7fe6c31","Type":"ContainerStarted","Data":"38d1340e4e8f4eb7fb1a5f25bb2ca814b6dd91128696108a65dd01bae70c2d0b"} Jul 10 20:58:46.317865 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:46.317837 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775f448-7k2mq" event={"ID":"8820d5ab-9032-40c5-801c-445420e8a285","Type":"ContainerStarted","Data":"2cd5aa0de3cb8703061dc42902bfbc6293c936ed31eaf7d7dc4957a0dae70bb7"} Jul 10 20:58:46.319093 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:46.319072 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" event={"ID":"77262f90-b294-41df-b9ca-7b76965a84f6","Type":"ContainerStarted","Data":"b94204186d14f16201a7a1b2bc3cc8b7a6208c02fdf9eec8e3913e0feb3ab03e"} Jul 10 20:58:46.320352 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:46.320331 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" event={"ID":"6d5632d4-a66d-4ee5-b796-9a2117e4e2b6","Type":"ContainerStarted","Data":"a39d0e8fa4d644be2fc1dd4c77136566b5d8b8e60058672d7fc850f97213b3ed"} Jul 10 20:58:46.364019 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:46.363971 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mpbf9" podStartSLOduration=17.188674052 podStartE2EDuration="25.363955406s" totalImagesPullingTime="8.175281354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:58:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:46.36346966 +0000 UTC m=+59.947165262" watchObservedRunningTime="2026-07-10 20:58:46.363955406 +0000 UTC m=+59.947651008" Jul 10 20:58:51.341581 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:51.341531 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" event={"ID":"77262f90-b294-41df-b9ca-7b76965a84f6","Type":"ContainerStarted","Data":"462fae4aac9ee7a3d9541fe8ee71889b2bed29f514919b359433d364b7426c38"} Jul 10 20:58:51.341581 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:51.341586 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" event={"ID":"77262f90-b294-41df-b9ca-7b76965a84f6","Type":"ContainerStarted","Data":"f1f154d01434cde57aa0bfae85b9d44a28031cf64cbe62a9b8c4eb788e4b83c2"} Jul 10 20:58:51.342100 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:51.341599 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" event={"ID":"77262f90-b294-41df-b9ca-7b76965a84f6","Type":"ContainerStarted","Data":"4f92e3d4df8f660b1ec951f1956685249a37bef0608c1ac394aa05c09a0ae099"} Jul 10 20:58:51.363313 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:51.363264 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bb6b95559-k5xlc" podStartSLOduration=3.155571236 podStartE2EDuration="7.363249386s" totalImagesPullingTime="4.20767815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:51.361862005 +0000 UTC m=+64.945557598" watchObservedRunningTime="2026-07-10 20:58:51.363249386 +0000 UTC m=+64.946944967" Jul 10 20:58:52.347315 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.347273 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775f448-7k2mq" event={"ID":"8820d5ab-9032-40c5-801c-445420e8a285","Type":"ContainerStarted","Data":"e43e529a9178f03840daa8ed7ac6aad82689c96bdb8da26e6687d8e6378f60b6"} Jul 10 20:58:52.364886 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.364826 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c775f448-7k2mq" podStartSLOduration=2.341341155 podStartE2EDuration="8.36480922s" totalImagesPullingTime="6.023468065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:52.3644294 +0000 UTC m=+65.948125003" watchObservedRunningTime="2026-07-10 20:58:52.36480922 +0000 UTC m=+65.948504822" Jul 10 20:58:52.758635 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.758598 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs\") pod \"network-metrics-daemon-lk7sj\" (UID: \"ca5ea89b-dd36-4208-b892-a9b1176c0234\") " pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:52.758841 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.758672 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:52.762654 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.762601 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 20:58:52.762783 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.762699 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 20:58:52.771368 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.771343 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94f8ad00-553c-47e0-8453-cdbe5b2a2b39-original-pull-secret\") pod \"global-pull-secret-syncer-9788m\" (UID: \"94f8ad00-553c-47e0-8453-cdbe5b2a2b39\") " pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:52.771502 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.771414 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca5ea89b-dd36-4208-b892-a9b1176c0234-metrics-certs\") pod \"network-metrics-daemon-lk7sj\" (UID: \"ca5ea89b-dd36-4208-b892-a9b1176c0234\") " pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:52.776834 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.776813 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8kn2h\"" Jul 10 20:58:52.785076 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.785054 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lk7sj" Jul 10 20:58:52.789615 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.789596 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9788m" Jul 10 20:58:52.859412 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.859366 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcjp8\" (UniqueName: \"kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8\") pod \"network-check-target-8lb77\" (UID: \"03703dd3-bccf-4be3-bd9d-68c093249f6c\") " pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:52.862969 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.862942 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 20:58:52.874023 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.874001 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 20:58:52.884012 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:52.883984 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcjp8\" (UniqueName: \"kubernetes.io/projected/03703dd3-bccf-4be3-bd9d-68c093249f6c-kube-api-access-bcjp8\") pod \"network-check-target-8lb77\" (UID: \"03703dd3-bccf-4be3-bd9d-68c093249f6c\") " pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:53.085613 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:53.085528 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4fk7p\"" Jul 10 20:58:53.092833 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:53.092800 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:53.816028 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:53.815961 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9788m"] Jul 10 20:58:53.819378 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:58:53.819349 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94f8ad00_553c_47e0_8453_cdbe5b2a2b39.slice/crio-aca8d0940920bd0f08129ebb938aece9d81fddaa9a6d996e1c38c7c2fa6f0458 WatchSource:0}: Error finding container aca8d0940920bd0f08129ebb938aece9d81fddaa9a6d996e1c38c7c2fa6f0458: Status 404 returned error can't find the container with id aca8d0940920bd0f08129ebb938aece9d81fddaa9a6d996e1c38c7c2fa6f0458 Jul 10 20:58:54.025032 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:54.024999 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lk7sj"] Jul 10 20:58:54.028626 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:54.028567 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8lb77"] Jul 10 20:58:54.029451 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:58:54.029421 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca5ea89b_dd36_4208_b892_a9b1176c0234.slice/crio-9268beb39b7ab159bf926148156edcd22af61969aa8b38909c921b76675695a9 WatchSource:0}: Error finding container 9268beb39b7ab159bf926148156edcd22af61969aa8b38909c921b76675695a9: Status 404 returned error can't find the container with id 9268beb39b7ab159bf926148156edcd22af61969aa8b38909c921b76675695a9 Jul 10 20:58:54.032223 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:58:54.032203 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03703dd3_bccf_4be3_bd9d_68c093249f6c.slice/crio-122e7381e775c7575e906489f0131b6c48ead60d08bc422cfd9991cfe5d7f972 WatchSource:0}: Error finding container 122e7381e775c7575e906489f0131b6c48ead60d08bc422cfd9991cfe5d7f972: Status 404 returned error can't find the container with id 122e7381e775c7575e906489f0131b6c48ead60d08bc422cfd9991cfe5d7f972 Jul 10 20:58:54.355177 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:54.355074 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8lb77" event={"ID":"03703dd3-bccf-4be3-bd9d-68c093249f6c","Type":"ContainerStarted","Data":"122e7381e775c7575e906489f0131b6c48ead60d08bc422cfd9991cfe5d7f972"} Jul 10 20:58:54.356693 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:54.356663 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" event={"ID":"6d5632d4-a66d-4ee5-b796-9a2117e4e2b6","Type":"ContainerStarted","Data":"b2c0cf24e9a0c6148f8b1c5d3710d558e2fb90c95d8d50abb18723583f5aee3a"} Jul 10 20:58:54.356880 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:54.356863 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" Jul 10 20:58:54.358003 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:54.357962 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lk7sj" event={"ID":"ca5ea89b-dd36-4208-b892-a9b1176c0234","Type":"ContainerStarted","Data":"9268beb39b7ab159bf926148156edcd22af61969aa8b38909c921b76675695a9"} Jul 10 20:58:54.359015 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:54.358988 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" Jul 10 20:58:54.359422 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:54.359386 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9788m" event={"ID":"94f8ad00-553c-47e0-8453-cdbe5b2a2b39","Type":"ContainerStarted","Data":"aca8d0940920bd0f08129ebb938aece9d81fddaa9a6d996e1c38c7c2fa6f0458"} Jul 10 20:58:54.391151 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:54.391018 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-86c6ff9d79-rh4vp" podStartSLOduration=2.938531986 podStartE2EDuration="10.390992807s" totalImagesPullingTime="7.452460821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:54.373794266 +0000 UTC m=+67.957489870" watchObservedRunningTime="2026-07-10 20:58:54.390992807 +0000 UTC m=+67.974688422" Jul 10 20:58:55.370546 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:55.370433 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lk7sj" event={"ID":"ca5ea89b-dd36-4208-b892-a9b1176c0234","Type":"ContainerStarted","Data":"c8daae4c79144c3304d73a3f10c5d5973ffc1c0baa4565bc08c61308dcf06d9e"} Jul 10 20:58:55.370546 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:55.370532 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lk7sj" event={"ID":"ca5ea89b-dd36-4208-b892-a9b1176c0234","Type":"ContainerStarted","Data":"ae4e942c2a8b4abbf4f8592e73364813070d6f5015385d69f731d2573c0cdd1d"} Jul 10 20:58:55.387335 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:55.387246 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lk7sj" podStartSLOduration=67.423775066 podStartE2EDuration="1m8.387226585s" totalImagesPullingTime="963.451519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:55.38673514 +0000 UTC m=+68.970430745" watchObservedRunningTime="2026-07-10 20:58:55.387226585 +0000 UTC m=+68.970922189" Jul 10 20:58:57.769682 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.769643 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pwwvt"] Jul 10 20:58:57.786316 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.786285 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:57.788999 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.788965 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 20:58:57.788999 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.788966 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 20:58:57.789883 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.789842 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 20:58:57.789883 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.789876 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 20:58:57.790072 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.789934 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 20:58:57.790072 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.789943 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jrfw6\"" Jul 10 20:58:57.790222 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.790207 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 20:58:57.898028 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.897989 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-tls\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:57.898295 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.898054 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d5896fa-3c3f-43d8-963f-5705467f838d-metrics-client-ca\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:57.898295 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.898089 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d5896fa-3c3f-43d8-963f-5705467f838d-sys\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:57.898295 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.898149 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:57.898295 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.898209 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:57.898295 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.898254 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5f89\" (UniqueName: \"kubernetes.io/projected/8d5896fa-3c3f-43d8-963f-5705467f838d-kube-api-access-m5f89\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:57.898572 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.898315 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-textfile\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:57.898572 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.898355 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-wtmp\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:57.898572 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.898389 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8d5896fa-3c3f-43d8-963f-5705467f838d-root\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:57.999810 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.999769 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-textfile\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:57.999810 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.999819 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-wtmp\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.000113 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.999851 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8d5896fa-3c3f-43d8-963f-5705467f838d-root\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.000113 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.999879 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-tls\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.000113 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.999911 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d5896fa-3c3f-43d8-963f-5705467f838d-metrics-client-ca\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.000113 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.999937 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d5896fa-3c3f-43d8-963f-5705467f838d-sys\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.000113 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:57.999966 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.000113 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:58.000002 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.000113 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:58.000048 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5f89\" (UniqueName: \"kubernetes.io/projected/8d5896fa-3c3f-43d8-963f-5705467f838d-kube-api-access-m5f89\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.000468 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:58.000145 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d5896fa-3c3f-43d8-963f-5705467f838d-sys\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.000527 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:58.000464 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-textfile\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.000726 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:58.000702 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8d5896fa-3c3f-43d8-963f-5705467f838d-root\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.000726 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:58.000721 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-wtmp\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.000915 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:58.000762 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d5896fa-3c3f-43d8-963f-5705467f838d-metrics-client-ca\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.000915 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:58.000794 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 10 20:58:58.000915 ip-10-0-131-227 kubenswrapper[2566]: E0710 20:58:58.000863 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-tls podName:8d5896fa-3c3f-43d8-963f-5705467f838d nodeName:}" failed. No retries permitted until 2026-07-10 20:58:58.500842361 +0000 UTC m=+72.084537953 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-tls") pod "node-exporter-pwwvt" (UID: "8d5896fa-3c3f-43d8-963f-5705467f838d") : secret "node-exporter-tls" not found Jul 10 20:58:58.001030 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:58.001008 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-accelerators-collector-config\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.003096 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:58.003074 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.009186 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:58.009155 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5f89\" (UniqueName: \"kubernetes.io/projected/8d5896fa-3c3f-43d8-963f-5705467f838d-kube-api-access-m5f89\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.503796 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:58.503758 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-tls\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.506608 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:58.506578 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d5896fa-3c3f-43d8-963f-5705467f838d-node-exporter-tls\") pod \"node-exporter-pwwvt\" (UID: \"8d5896fa-3c3f-43d8-963f-5705467f838d\") " pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.697009 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:58.696938 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pwwvt" Jul 10 20:58:58.706268 ip-10-0-131-227 kubenswrapper[2566]: W0710 20:58:58.706244 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d5896fa_3c3f_43d8_963f_5705467f838d.slice/crio-f620602954d8f38736a10a6667ca289c6de47907e54f88f353027dd4b9d1880c WatchSource:0}: Error finding container f620602954d8f38736a10a6667ca289c6de47907e54f88f353027dd4b9d1880c: Status 404 returned error can't find the container with id f620602954d8f38736a10a6667ca289c6de47907e54f88f353027dd4b9d1880c Jul 10 20:58:59.386569 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:59.386526 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8lb77" event={"ID":"03703dd3-bccf-4be3-bd9d-68c093249f6c","Type":"ContainerStarted","Data":"141ddcfa08ff16c940a366b738dce9f5ded0ffdda116e8f4fa66ff7521fe007f"} Jul 10 20:58:59.387003 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:59.386702 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:58:59.388208 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:59.388183 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9788m" event={"ID":"94f8ad00-553c-47e0-8453-cdbe5b2a2b39","Type":"ContainerStarted","Data":"f6c81737c3c6e5c656806fd1a7ced3eba66cf4997f9d823fa3bd8e50ecea6432"} Jul 10 20:58:59.389537 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:59.389486 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwwvt" event={"ID":"8d5896fa-3c3f-43d8-963f-5705467f838d","Type":"ContainerStarted","Data":"f620602954d8f38736a10a6667ca289c6de47907e54f88f353027dd4b9d1880c"} Jul 10 20:58:59.406090 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:59.406036 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8lb77" podStartSLOduration=67.85014737 podStartE2EDuration="1m12.406022233s" totalImagesPullingTime="4.555874863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:59.404756482 +0000 UTC m=+72.988452110" watchObservedRunningTime="2026-07-10 20:58:59.406022233 +0000 UTC m=+72.989717835" Jul 10 20:58:59.431356 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:58:59.431306 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9788m" podStartSLOduration=67.659271589 podStartE2EDuration="1m12.431291332s" totalImagesPullingTime="4.772019743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:58:59.430294739 +0000 UTC m=+73.013990342" watchObservedRunningTime="2026-07-10 20:58:59.431291332 +0000 UTC m=+73.014986937" Jul 10 20:59:00.395554 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:59:00.395516 2566 generic.go:356] "Generic (PLEG): container finished" podID="8d5896fa-3c3f-43d8-963f-5705467f838d" containerID="2180a98266a977aa62fad06e640f138ec3d318a7133fc1bd6a061c0158b49d12" exitCode=0 Jul 10 20:59:00.396034 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:59:00.395610 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwwvt" event={"ID":"8d5896fa-3c3f-43d8-963f-5705467f838d","Type":"ContainerDied","Data":"2180a98266a977aa62fad06e640f138ec3d318a7133fc1bd6a061c0158b49d12"} Jul 10 20:59:01.404587 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:59:01.404552 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwwvt" event={"ID":"8d5896fa-3c3f-43d8-963f-5705467f838d","Type":"ContainerStarted","Data":"95aeaf68a2c2aeac382d6701f07e3547b8785d1e4faebeabdab56e1bb9858899"} Jul 10 20:59:01.404972 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:59:01.404631 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pwwvt" event={"ID":"8d5896fa-3c3f-43d8-963f-5705467f838d","Type":"ContainerStarted","Data":"4f4ebbcd11e5e7b8da5464d4a4cbe056c00c8c57ae457302bc8709a298527184"} Jul 10 20:59:01.438832 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:59:01.438782 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pwwvt" podStartSLOduration=3.371159501 podStartE2EDuration="4.438766186s" totalImagesPullingTime="1.004560221s" totalInitContainerRuntime="63.046464ms" isStatefulPod=true podCreationTimestamp="2026-07-10 20:58:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:59:01.438212873 +0000 UTC m=+75.021908474" watchObservedRunningTime="2026-07-10 20:59:01.438766186 +0000 UTC m=+75.022461788" Jul 10 20:59:30.398367 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:59:30.398334 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8lb77" Jul 10 20:59:45.330383 ip-10-0-131-227 kubenswrapper[2566]: I0710 20:59:45.330352 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-84gxc_69b2e978-68f0-4f4d-acf2-1cc9ef90cd28/serve-healthcheck-canary/0.log" Jul 10 21:02:10.317008 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:10.316928 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jcxfk"] Jul 10 21:02:10.319060 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:10.319043 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jcxfk" Jul 10 21:02:10.321445 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:10.321420 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 21:02:10.322226 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:10.322210 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-6lrrc\"" Jul 10 21:02:10.322321 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:10.322235 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 21:02:10.327971 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:10.327944 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jcxfk"] Jul 10 21:02:10.401193 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:10.401140 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhrv7\" (UniqueName: \"kubernetes.io/projected/cceb8422-5458-45ce-9aff-8cdb73bc4b0c-kube-api-access-lhrv7\") pod \"cert-manager-55b4d954d5-jcxfk\" (UID: \"cceb8422-5458-45ce-9aff-8cdb73bc4b0c\") " pod="cert-manager/cert-manager-55b4d954d5-jcxfk" Jul 10 21:02:10.401193 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:10.401213 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cceb8422-5458-45ce-9aff-8cdb73bc4b0c-bound-sa-token\") pod \"cert-manager-55b4d954d5-jcxfk\" (UID: \"cceb8422-5458-45ce-9aff-8cdb73bc4b0c\") " pod="cert-manager/cert-manager-55b4d954d5-jcxfk" Jul 10 21:02:10.502325 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:10.502282 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhrv7\" (UniqueName: \"kubernetes.io/projected/cceb8422-5458-45ce-9aff-8cdb73bc4b0c-kube-api-access-lhrv7\") pod \"cert-manager-55b4d954d5-jcxfk\" (UID: \"cceb8422-5458-45ce-9aff-8cdb73bc4b0c\") " pod="cert-manager/cert-manager-55b4d954d5-jcxfk" Jul 10 21:02:10.502571 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:10.502338 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cceb8422-5458-45ce-9aff-8cdb73bc4b0c-bound-sa-token\") pod \"cert-manager-55b4d954d5-jcxfk\" (UID: \"cceb8422-5458-45ce-9aff-8cdb73bc4b0c\") " pod="cert-manager/cert-manager-55b4d954d5-jcxfk" Jul 10 21:02:10.511355 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:10.511330 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cceb8422-5458-45ce-9aff-8cdb73bc4b0c-bound-sa-token\") pod \"cert-manager-55b4d954d5-jcxfk\" (UID: \"cceb8422-5458-45ce-9aff-8cdb73bc4b0c\") " pod="cert-manager/cert-manager-55b4d954d5-jcxfk" Jul 10 21:02:10.511507 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:10.511466 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhrv7\" (UniqueName: \"kubernetes.io/projected/cceb8422-5458-45ce-9aff-8cdb73bc4b0c-kube-api-access-lhrv7\") pod \"cert-manager-55b4d954d5-jcxfk\" (UID: \"cceb8422-5458-45ce-9aff-8cdb73bc4b0c\") " pod="cert-manager/cert-manager-55b4d954d5-jcxfk" Jul 10 21:02:10.628705 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:10.628607 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jcxfk" Jul 10 21:02:10.755076 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:10.755046 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jcxfk"] Jul 10 21:02:10.758739 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:02:10.758708 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcceb8422_5458_45ce_9aff_8cdb73bc4b0c.slice/crio-60253def64725f31e5922013acd7c52cef7dfa9ec81d01112abced3617587e69 WatchSource:0}: Error finding container 60253def64725f31e5922013acd7c52cef7dfa9ec81d01112abced3617587e69: Status 404 returned error can't find the container with id 60253def64725f31e5922013acd7c52cef7dfa9ec81d01112abced3617587e69 Jul 10 21:02:11.016660 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:11.016627 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jcxfk" event={"ID":"cceb8422-5458-45ce-9aff-8cdb73bc4b0c","Type":"ContainerStarted","Data":"60253def64725f31e5922013acd7c52cef7dfa9ec81d01112abced3617587e69"} Jul 10 21:02:14.029538 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:14.029505 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jcxfk" event={"ID":"cceb8422-5458-45ce-9aff-8cdb73bc4b0c","Type":"ContainerStarted","Data":"e139d530a614272ec607c7c242e9eb3feb22cb990e99c8c7c8702e56fa7399a4"} Jul 10 21:02:14.044873 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:14.044826 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-jcxfk" podStartSLOduration=1.008740828 podStartE2EDuration="4.044811003s" totalImagesPullingTime="3.036070175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:02:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:02:14.043795029 +0000 UTC m=+267.627490632" watchObservedRunningTime="2026-07-10 21:02:14.044811003 +0000 UTC m=+267.628506604" Jul 10 21:02:33.951654 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:33.951621 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8"] Jul 10 21:02:33.956553 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:33.956057 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:33.960928 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:33.960904 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 21:02:33.961063 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:33.960942 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 21:02:33.961063 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:33.960954 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 21:02:33.961063 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:33.960998 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-574bc\"" Jul 10 21:02:33.961285 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:33.961271 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 21:02:33.980140 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:33.980107 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8"] Jul 10 21:02:34.076034 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.075993 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74x26\" (UniqueName: \"kubernetes.io/projected/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-kube-api-access-74x26\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.076210 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.076053 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-tmp\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.076210 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.076112 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.076210 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.076131 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-webhook-cert\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.076210 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.076150 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-manifests\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.177195 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.177157 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-tmp\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.177195 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.177201 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.177486 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.177221 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-webhook-cert\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.177486 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.177237 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-manifests\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.177486 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.177269 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74x26\" (UniqueName: \"kubernetes.io/projected/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-kube-api-access-74x26\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.177742 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.177720 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-manifests\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.179644 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.179612 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-tmp\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.179774 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.179707 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-webhook-cert\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.179774 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.179713 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.185777 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.185753 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74x26\" (UniqueName: \"kubernetes.io/projected/4ffb42af-fb8a-4ea8-920e-3a4ce850e983-kube-api-access-74x26\") pod \"opendatahub-operator-controller-manager-58b8cc5f97-k8rt8\" (UID: \"4ffb42af-fb8a-4ea8-920e-3a4ce850e983\") " pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.266665 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.266624 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:34.397916 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:34.397883 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8"] Jul 10 21:02:34.401417 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:02:34.401371 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ffb42af_fb8a_4ea8_920e_3a4ce850e983.slice/crio-159b5795f10cb96830ff2d3d4f526138344538a522441adc01425e9b61b85fd7 WatchSource:0}: Error finding container 159b5795f10cb96830ff2d3d4f526138344538a522441adc01425e9b61b85fd7: Status 404 returned error can't find the container with id 159b5795f10cb96830ff2d3d4f526138344538a522441adc01425e9b61b85fd7 Jul 10 21:02:35.099153 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:35.099114 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" event={"ID":"4ffb42af-fb8a-4ea8-920e-3a4ce850e983","Type":"ContainerStarted","Data":"159b5795f10cb96830ff2d3d4f526138344538a522441adc01425e9b61b85fd7"} Jul 10 21:02:38.112678 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:38.112637 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" event={"ID":"4ffb42af-fb8a-4ea8-920e-3a4ce850e983","Type":"ContainerStarted","Data":"5c0d1ebc7b0a52eb5ddd20dac3b3d6888d9791bb72420cabda47e909c78c6486"} Jul 10 21:02:39.116845 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:39.116808 2566 generic.go:356] "Generic (PLEG): container finished" podID="4ffb42af-fb8a-4ea8-920e-3a4ce850e983" containerID="5c0d1ebc7b0a52eb5ddd20dac3b3d6888d9791bb72420cabda47e909c78c6486" exitCode=0 Jul 10 21:02:39.117246 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:39.116888 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" event={"ID":"4ffb42af-fb8a-4ea8-920e-3a4ce850e983","Type":"ContainerDied","Data":"5c0d1ebc7b0a52eb5ddd20dac3b3d6888d9791bb72420cabda47e909c78c6486"} Jul 10 21:02:40.125556 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:40.125517 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" event={"ID":"4ffb42af-fb8a-4ea8-920e-3a4ce850e983","Type":"ContainerStarted","Data":"87aedbbacaecb14917589de3e76f127dabf94572995e1e3a17cf3ee87080d23d"} Jul 10 21:02:40.126161 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:40.125644 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:40.148555 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:40.148488 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" podStartSLOduration=2.544161837 podStartE2EDuration="7.148464792s" totalImagesPullingTime="3.537614142s" totalInitContainerRuntime="1.066688813s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:02:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:02:40.146262221 +0000 UTC m=+293.729957823" watchObservedRunningTime="2026-07-10 21:02:40.148464792 +0000 UTC m=+293.732160387" Jul 10 21:02:46.853371 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:46.853343 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 21:02:51.132074 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:51.132046 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58b8cc5f97-k8rt8" Jul 10 21:02:53.794138 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.794104 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx"] Jul 10 21:02:53.797543 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.797523 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" Jul 10 21:02:53.800883 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.800854 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 21:02:53.800883 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.800880 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cfxln\"" Jul 10 21:02:53.801624 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.801596 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 21:02:53.801766 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.801648 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 21:02:53.801766 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.801656 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 21:02:53.824638 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.824610 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx"] Jul 10 21:02:53.884461 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.884417 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j5pjd"] Jul 10 21:02:53.887938 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.887917 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-j5pjd" Jul 10 21:02:53.890656 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.890626 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 10 21:02:53.890656 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.890638 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-6s9bn\"" Jul 10 21:02:53.905823 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.905793 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j5pjd"] Jul 10 21:02:53.931445 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.931409 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b614d91b-dc61-4bb6-954f-0307f7da3509-tmp\") pod \"kube-auth-proxy-cc6fcfbb4-4kzrx\" (UID: \"b614d91b-dc61-4bb6-954f-0307f7da3509\") " pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" Jul 10 21:02:53.931610 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.931473 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hshvq\" (UniqueName: \"kubernetes.io/projected/b614d91b-dc61-4bb6-954f-0307f7da3509-kube-api-access-hshvq\") pod \"kube-auth-proxy-cc6fcfbb4-4kzrx\" (UID: \"b614d91b-dc61-4bb6-954f-0307f7da3509\") " pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" Jul 10 21:02:53.931610 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:53.931498 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b614d91b-dc61-4bb6-954f-0307f7da3509-tls-certs\") pod \"kube-auth-proxy-cc6fcfbb4-4kzrx\" (UID: \"b614d91b-dc61-4bb6-954f-0307f7da3509\") " pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" Jul 10 21:02:54.032856 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.032818 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81dce030-62c0-4144-8b18-9e8642722fbc-cert\") pod \"odh-model-controller-7998c67dc4-j5pjd\" (UID: \"81dce030-62c0-4144-8b18-9e8642722fbc\") " pod="opendatahub/odh-model-controller-7998c67dc4-j5pjd" Jul 10 21:02:54.033047 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.032964 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b614d91b-dc61-4bb6-954f-0307f7da3509-tmp\") pod \"kube-auth-proxy-cc6fcfbb4-4kzrx\" (UID: \"b614d91b-dc61-4bb6-954f-0307f7da3509\") " pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" Jul 10 21:02:54.033047 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.033016 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hshvq\" (UniqueName: \"kubernetes.io/projected/b614d91b-dc61-4bb6-954f-0307f7da3509-kube-api-access-hshvq\") pod \"kube-auth-proxy-cc6fcfbb4-4kzrx\" (UID: \"b614d91b-dc61-4bb6-954f-0307f7da3509\") " pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" Jul 10 21:02:54.033155 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.033050 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7bh9\" (UniqueName: \"kubernetes.io/projected/81dce030-62c0-4144-8b18-9e8642722fbc-kube-api-access-m7bh9\") pod \"odh-model-controller-7998c67dc4-j5pjd\" (UID: \"81dce030-62c0-4144-8b18-9e8642722fbc\") " pod="opendatahub/odh-model-controller-7998c67dc4-j5pjd" Jul 10 21:02:54.033155 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.033084 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b614d91b-dc61-4bb6-954f-0307f7da3509-tls-certs\") pod \"kube-auth-proxy-cc6fcfbb4-4kzrx\" (UID: \"b614d91b-dc61-4bb6-954f-0307f7da3509\") " pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" Jul 10 21:02:54.033272 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:02:54.033205 2566 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 10 21:02:54.033324 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:02:54.033278 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b614d91b-dc61-4bb6-954f-0307f7da3509-tls-certs podName:b614d91b-dc61-4bb6-954f-0307f7da3509 nodeName:}" failed. No retries permitted until 2026-07-10 21:02:54.533254975 +0000 UTC m=+308.116950558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b614d91b-dc61-4bb6-954f-0307f7da3509-tls-certs") pod "kube-auth-proxy-cc6fcfbb4-4kzrx" (UID: "b614d91b-dc61-4bb6-954f-0307f7da3509") : secret "kube-auth-proxy-tls" not found Jul 10 21:02:54.035360 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.035329 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b614d91b-dc61-4bb6-954f-0307f7da3509-tmp\") pod \"kube-auth-proxy-cc6fcfbb4-4kzrx\" (UID: \"b614d91b-dc61-4bb6-954f-0307f7da3509\") " pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" Jul 10 21:02:54.044521 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.044443 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hshvq\" (UniqueName: \"kubernetes.io/projected/b614d91b-dc61-4bb6-954f-0307f7da3509-kube-api-access-hshvq\") pod \"kube-auth-proxy-cc6fcfbb4-4kzrx\" (UID: \"b614d91b-dc61-4bb6-954f-0307f7da3509\") " pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" Jul 10 21:02:54.134531 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.134496 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7bh9\" (UniqueName: \"kubernetes.io/projected/81dce030-62c0-4144-8b18-9e8642722fbc-kube-api-access-m7bh9\") pod \"odh-model-controller-7998c67dc4-j5pjd\" (UID: \"81dce030-62c0-4144-8b18-9e8642722fbc\") " pod="opendatahub/odh-model-controller-7998c67dc4-j5pjd" Jul 10 21:02:54.134717 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.134561 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81dce030-62c0-4144-8b18-9e8642722fbc-cert\") pod \"odh-model-controller-7998c67dc4-j5pjd\" (UID: \"81dce030-62c0-4144-8b18-9e8642722fbc\") " pod="opendatahub/odh-model-controller-7998c67dc4-j5pjd" Jul 10 21:02:54.137458 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.137427 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81dce030-62c0-4144-8b18-9e8642722fbc-cert\") pod \"odh-model-controller-7998c67dc4-j5pjd\" (UID: \"81dce030-62c0-4144-8b18-9e8642722fbc\") " pod="opendatahub/odh-model-controller-7998c67dc4-j5pjd" Jul 10 21:02:54.142560 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.142533 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7bh9\" (UniqueName: \"kubernetes.io/projected/81dce030-62c0-4144-8b18-9e8642722fbc-kube-api-access-m7bh9\") pod \"odh-model-controller-7998c67dc4-j5pjd\" (UID: \"81dce030-62c0-4144-8b18-9e8642722fbc\") " pod="opendatahub/odh-model-controller-7998c67dc4-j5pjd" Jul 10 21:02:54.199384 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.199338 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-j5pjd" Jul 10 21:02:54.339819 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.339729 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j5pjd"] Jul 10 21:02:54.343083 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:02:54.343052 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81dce030_62c0_4144_8b18_9e8642722fbc.slice/crio-6d0a53d5345fb4341837d93d33dece61563e99a7406c7d9539b0f6269c9faf9b WatchSource:0}: Error finding container 6d0a53d5345fb4341837d93d33dece61563e99a7406c7d9539b0f6269c9faf9b: Status 404 returned error can't find the container with id 6d0a53d5345fb4341837d93d33dece61563e99a7406c7d9539b0f6269c9faf9b Jul 10 21:02:54.344477 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.344457 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 21:02:54.538481 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.538445 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b614d91b-dc61-4bb6-954f-0307f7da3509-tls-certs\") pod \"kube-auth-proxy-cc6fcfbb4-4kzrx\" (UID: \"b614d91b-dc61-4bb6-954f-0307f7da3509\") " pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" Jul 10 21:02:54.540863 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.540829 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b614d91b-dc61-4bb6-954f-0307f7da3509-tls-certs\") pod \"kube-auth-proxy-cc6fcfbb4-4kzrx\" (UID: \"b614d91b-dc61-4bb6-954f-0307f7da3509\") " pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" Jul 10 21:02:54.675211 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.675129 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6qmtk"] Jul 10 21:02:54.679315 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.679295 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6qmtk" Jul 10 21:02:54.682143 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.682120 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 10 21:02:54.682622 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.682602 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-d6c6n\"" Jul 10 21:02:54.707876 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.707833 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" Jul 10 21:02:54.754476 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.754446 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6qmtk"] Jul 10 21:02:54.842306 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.842265 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6qrn\" (UniqueName: \"kubernetes.io/projected/c6cf8967-278c-40be-bd75-e0af1e751c6b-kube-api-access-f6qrn\") pod \"model-serving-api-54d89bd5fc-6qmtk\" (UID: \"c6cf8967-278c-40be-bd75-e0af1e751c6b\") " pod="opendatahub/model-serving-api-54d89bd5fc-6qmtk" Jul 10 21:02:54.842815 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.842436 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6cf8967-278c-40be-bd75-e0af1e751c6b-tls-certs\") pod \"model-serving-api-54d89bd5fc-6qmtk\" (UID: \"c6cf8967-278c-40be-bd75-e0af1e751c6b\") " pod="opendatahub/model-serving-api-54d89bd5fc-6qmtk" Jul 10 21:02:54.860016 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.859835 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx"] Jul 10 21:02:54.862474 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:02:54.862446 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb614d91b_dc61_4bb6_954f_0307f7da3509.slice/crio-ea4f3d68dd884be7b0fb07f1500fe8362939edbe4d34c8ea695aac31eb7cfed9 WatchSource:0}: Error finding container ea4f3d68dd884be7b0fb07f1500fe8362939edbe4d34c8ea695aac31eb7cfed9: Status 404 returned error can't find the container with id ea4f3d68dd884be7b0fb07f1500fe8362939edbe4d34c8ea695aac31eb7cfed9 Jul 10 21:02:54.943877 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.943783 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6cf8967-278c-40be-bd75-e0af1e751c6b-tls-certs\") pod \"model-serving-api-54d89bd5fc-6qmtk\" (UID: \"c6cf8967-278c-40be-bd75-e0af1e751c6b\") " pod="opendatahub/model-serving-api-54d89bd5fc-6qmtk" Jul 10 21:02:54.943877 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.943839 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6qrn\" (UniqueName: \"kubernetes.io/projected/c6cf8967-278c-40be-bd75-e0af1e751c6b-kube-api-access-f6qrn\") pod \"model-serving-api-54d89bd5fc-6qmtk\" (UID: \"c6cf8967-278c-40be-bd75-e0af1e751c6b\") " pod="opendatahub/model-serving-api-54d89bd5fc-6qmtk" Jul 10 21:02:54.944107 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:02:54.943955 2566 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 10 21:02:54.944107 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:02:54.944037 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6cf8967-278c-40be-bd75-e0af1e751c6b-tls-certs podName:c6cf8967-278c-40be-bd75-e0af1e751c6b nodeName:}" failed. No retries permitted until 2026-07-10 21:02:55.444014737 +0000 UTC m=+309.027710335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/c6cf8967-278c-40be-bd75-e0af1e751c6b-tls-certs") pod "model-serving-api-54d89bd5fc-6qmtk" (UID: "c6cf8967-278c-40be-bd75-e0af1e751c6b") : secret "model-serving-api-tls" not found Jul 10 21:02:54.955929 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:54.955899 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6qrn\" (UniqueName: \"kubernetes.io/projected/c6cf8967-278c-40be-bd75-e0af1e751c6b-kube-api-access-f6qrn\") pod \"model-serving-api-54d89bd5fc-6qmtk\" (UID: \"c6cf8967-278c-40be-bd75-e0af1e751c6b\") " pod="opendatahub/model-serving-api-54d89bd5fc-6qmtk" Jul 10 21:02:55.177054 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.177016 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" event={"ID":"b614d91b-dc61-4bb6-954f-0307f7da3509","Type":"ContainerStarted","Data":"ea4f3d68dd884be7b0fb07f1500fe8362939edbe4d34c8ea695aac31eb7cfed9"} Jul 10 21:02:55.178197 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.178168 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j5pjd" event={"ID":"81dce030-62c0-4144-8b18-9e8642722fbc","Type":"ContainerStarted","Data":"6d0a53d5345fb4341837d93d33dece61563e99a7406c7d9539b0f6269c9faf9b"} Jul 10 21:02:55.285750 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.285717 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qftrp"] Jul 10 21:02:55.290313 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.290295 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.292612 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.292592 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 21:02:55.292847 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.292832 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 21:02:55.292929 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.292892 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fp5ct\"" Jul 10 21:02:55.292929 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.292894 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 21:02:55.293143 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.293125 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 21:02:55.303729 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.303702 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qftrp"] Jul 10 21:02:55.448825 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.448780 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.448825 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.448831 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.449088 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.448851 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhr55\" (UniqueName: \"kubernetes.io/projected/8f0b4a83-ed9b-4750-9662-9b7472258bd7-kube-api-access-nhr55\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.449088 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.448916 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-cacerts\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.449088 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.448960 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.449088 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.449000 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0b4a83-ed9b-4750-9662-9b7472258bd7-local-certs\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.449088 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.449028 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-token\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.449088 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.449061 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6cf8967-278c-40be-bd75-e0af1e751c6b-tls-certs\") pod \"model-serving-api-54d89bd5fc-6qmtk\" (UID: \"c6cf8967-278c-40be-bd75-e0af1e751c6b\") " pod="opendatahub/model-serving-api-54d89bd5fc-6qmtk" Jul 10 21:02:55.451611 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.451575 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6cf8967-278c-40be-bd75-e0af1e751c6b-tls-certs\") pod \"model-serving-api-54d89bd5fc-6qmtk\" (UID: \"c6cf8967-278c-40be-bd75-e0af1e751c6b\") " pod="opendatahub/model-serving-api-54d89bd5fc-6qmtk" Jul 10 21:02:55.549822 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.549720 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.549822 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.549795 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.550043 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.549827 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhr55\" (UniqueName: \"kubernetes.io/projected/8f0b4a83-ed9b-4750-9662-9b7472258bd7-kube-api-access-nhr55\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.550043 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.549865 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-cacerts\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.550043 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.549890 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.550043 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.549939 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0b4a83-ed9b-4750-9662-9b7472258bd7-local-certs\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.550043 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.549974 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-token\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.550871 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.550536 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.553336 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.553290 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.554694 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.554666 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0b4a83-ed9b-4750-9662-9b7472258bd7-local-certs\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.555304 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.555261 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.555439 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.555301 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-cacerts\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.559807 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.559762 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-token\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.561946 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.561921 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhr55\" (UniqueName: \"kubernetes.io/projected/8f0b4a83-ed9b-4750-9662-9b7472258bd7-kube-api-access-nhr55\") pod \"istiod-openshift-gateway-7574459656-qftrp\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.589902 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.589832 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6qmtk" Jul 10 21:02:55.603109 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.603072 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:02:55.791952 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.791918 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6qmtk"] Jul 10 21:02:55.795937 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:02:55.795899 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6cf8967_278c_40be_bd75_e0af1e751c6b.slice/crio-eff691a8781baa35fdfacbbd57aeb44c6884deed0460ae786dccc911fd999291 WatchSource:0}: Error finding container eff691a8781baa35fdfacbbd57aeb44c6884deed0460ae786dccc911fd999291: Status 404 returned error can't find the container with id eff691a8781baa35fdfacbbd57aeb44c6884deed0460ae786dccc911fd999291 Jul 10 21:02:55.819309 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:55.819278 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qftrp"] Jul 10 21:02:55.822852 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:02:55.822811 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f0b4a83_ed9b_4750_9662_9b7472258bd7.slice/crio-6d81993a078e724988e374f8e513c75e068ec9b081446b767927422cd2ceaccd WatchSource:0}: Error finding container 6d81993a078e724988e374f8e513c75e068ec9b081446b767927422cd2ceaccd: Status 404 returned error can't find the container with id 6d81993a078e724988e374f8e513c75e068ec9b081446b767927422cd2ceaccd Jul 10 21:02:56.184803 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:56.184707 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6qmtk" event={"ID":"c6cf8967-278c-40be-bd75-e0af1e751c6b","Type":"ContainerStarted","Data":"eff691a8781baa35fdfacbbd57aeb44c6884deed0460ae786dccc911fd999291"} Jul 10 21:02:56.186189 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:02:56.186152 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" event={"ID":"8f0b4a83-ed9b-4750-9662-9b7472258bd7","Type":"ContainerStarted","Data":"6d81993a078e724988e374f8e513c75e068ec9b081446b767927422cd2ceaccd"} Jul 10 21:03:00.633788 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:00.633743 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:03:00.634093 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:00.633831 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 10 21:03:01.209197 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:01.209159 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" event={"ID":"b614d91b-dc61-4bb6-954f-0307f7da3509","Type":"ContainerStarted","Data":"169e300da9708b3c5c6b030ef99e002c52bafb7a45882ef776e4fa8a035594cf"} Jul 10 21:03:01.211203 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:01.211164 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j5pjd" event={"ID":"81dce030-62c0-4144-8b18-9e8642722fbc","Type":"ContainerStarted","Data":"900d03dd1a44776b9f12fc6a7574f76cdfceda539ea372f20a7c0c422a84b253"} Jul 10 21:03:01.211344 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:01.211260 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-j5pjd" Jul 10 21:03:01.213178 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:01.213146 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6qmtk" event={"ID":"c6cf8967-278c-40be-bd75-e0af1e751c6b","Type":"ContainerStarted","Data":"81cca8ce7bca6624475e6b12965800640bb2d776f1385c39cbd73072c27f8aa9"} Jul 10 21:03:01.213326 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:01.213217 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-6qmtk" Jul 10 21:03:01.215260 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:01.215231 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" event={"ID":"8f0b4a83-ed9b-4750-9662-9b7472258bd7","Type":"ContainerStarted","Data":"52db59590f359843daadf63db8400b27e32dedd5baebca07b39cbd638aee33da"} Jul 10 21:03:01.215410 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:01.215383 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:03:01.217150 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:01.217123 2566 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-qftrp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 21:03:01.217254 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:01.217190 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" podUID="8f0b4a83-ed9b-4750-9662-9b7472258bd7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 21:03:01.229179 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:01.229131 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-cc6fcfbb4-4kzrx" podStartSLOduration=2.466448432 podStartE2EDuration="8.2291158s" totalImagesPullingTime="5.762667368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:03:01.227914772 +0000 UTC m=+314.811610375" watchObservedRunningTime="2026-07-10 21:03:01.2291158 +0000 UTC m=+314.812811402" Jul 10 21:03:01.244154 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:01.244066 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-j5pjd" podStartSLOduration=1.961771459 podStartE2EDuration="8.244049041s" totalImagesPullingTime="6.282277582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:03:01.242652306 +0000 UTC m=+314.826347910" watchObservedRunningTime="2026-07-10 21:03:01.244049041 +0000 UTC m=+314.827744642" Jul 10 21:03:01.261352 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:01.261302 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" podStartSLOduration=1.453771342 podStartE2EDuration="6.261286499s" totalImagesPullingTime="4.807515157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:02:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:03:01.260285716 +0000 UTC m=+314.843981320" watchObservedRunningTime="2026-07-10 21:03:01.261286499 +0000 UTC m=+314.844982102" Jul 10 21:03:01.276993 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:01.276929 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-6qmtk" podStartSLOduration=2.449743173 podStartE2EDuration="7.276912614s" totalImagesPullingTime="4.827169441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:03:01.276204652 +0000 UTC m=+314.859900255" watchObservedRunningTime="2026-07-10 21:03:01.276912614 +0000 UTC m=+314.860608215" Jul 10 21:03:02.220181 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:02.220145 2566 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-qftrp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 21:03:02.220592 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:02.220203 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" podUID="8f0b4a83-ed9b-4750-9662-9b7472258bd7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 21:03:05.220548 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.220514 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:03:05.360965 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.360929 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7plcl"] Jul 10 21:03:05.364759 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.364726 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7plcl" Jul 10 21:03:05.367728 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.367686 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 10 21:03:05.367728 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.367703 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-k6mln\"" Jul 10 21:03:05.372700 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.372417 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn"] Jul 10 21:03:05.375618 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.375596 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn" Jul 10 21:03:05.379275 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.379254 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 10 21:03:05.379477 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.379454 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7plcl"] Jul 10 21:03:05.379604 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.379255 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wqh9m\"" Jul 10 21:03:05.397820 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.397792 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn"] Jul 10 21:03:05.528518 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.528433 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqf8r\" (UniqueName: \"kubernetes.io/projected/50226488-7e48-4dc6-b874-0f373bd4200f-kube-api-access-nqf8r\") pod \"llmisvc-controller-manager-777c6bd7d5-22bpn\" (UID: \"50226488-7e48-4dc6-b874-0f373bd4200f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn" Jul 10 21:03:05.528518 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.528469 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ac19b08-450e-40f9-85b0-48e5a3e39174-cert\") pod \"kserve-controller-manager-d6f4b65b8-7plcl\" (UID: \"8ac19b08-450e-40f9-85b0-48e5a3e39174\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7plcl" Jul 10 21:03:05.528518 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.528488 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50226488-7e48-4dc6-b874-0f373bd4200f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-22bpn\" (UID: \"50226488-7e48-4dc6-b874-0f373bd4200f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn" Jul 10 21:03:05.528736 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.528562 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xwnv\" (UniqueName: \"kubernetes.io/projected/8ac19b08-450e-40f9-85b0-48e5a3e39174-kube-api-access-7xwnv\") pod \"kserve-controller-manager-d6f4b65b8-7plcl\" (UID: \"8ac19b08-450e-40f9-85b0-48e5a3e39174\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7plcl" Jul 10 21:03:05.629738 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.629705 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqf8r\" (UniqueName: \"kubernetes.io/projected/50226488-7e48-4dc6-b874-0f373bd4200f-kube-api-access-nqf8r\") pod \"llmisvc-controller-manager-777c6bd7d5-22bpn\" (UID: \"50226488-7e48-4dc6-b874-0f373bd4200f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn" Jul 10 21:03:05.629738 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.629742 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ac19b08-450e-40f9-85b0-48e5a3e39174-cert\") pod \"kserve-controller-manager-d6f4b65b8-7plcl\" (UID: \"8ac19b08-450e-40f9-85b0-48e5a3e39174\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7plcl" Jul 10 21:03:05.629956 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.629765 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50226488-7e48-4dc6-b874-0f373bd4200f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-22bpn\" (UID: \"50226488-7e48-4dc6-b874-0f373bd4200f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn" Jul 10 21:03:05.629956 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.629792 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xwnv\" (UniqueName: \"kubernetes.io/projected/8ac19b08-450e-40f9-85b0-48e5a3e39174-kube-api-access-7xwnv\") pod \"kserve-controller-manager-d6f4b65b8-7plcl\" (UID: \"8ac19b08-450e-40f9-85b0-48e5a3e39174\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7plcl" Jul 10 21:03:05.632295 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.632262 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50226488-7e48-4dc6-b874-0f373bd4200f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-22bpn\" (UID: \"50226488-7e48-4dc6-b874-0f373bd4200f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn" Jul 10 21:03:05.632436 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.632262 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ac19b08-450e-40f9-85b0-48e5a3e39174-cert\") pod \"kserve-controller-manager-d6f4b65b8-7plcl\" (UID: \"8ac19b08-450e-40f9-85b0-48e5a3e39174\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7plcl" Jul 10 21:03:05.649003 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.648971 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xwnv\" (UniqueName: \"kubernetes.io/projected/8ac19b08-450e-40f9-85b0-48e5a3e39174-kube-api-access-7xwnv\") pod \"kserve-controller-manager-d6f4b65b8-7plcl\" (UID: \"8ac19b08-450e-40f9-85b0-48e5a3e39174\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7plcl" Jul 10 21:03:05.649541 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.649521 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqf8r\" (UniqueName: \"kubernetes.io/projected/50226488-7e48-4dc6-b874-0f373bd4200f-kube-api-access-nqf8r\") pod \"llmisvc-controller-manager-777c6bd7d5-22bpn\" (UID: \"50226488-7e48-4dc6-b874-0f373bd4200f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn" Jul 10 21:03:05.676453 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.676391 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7plcl" Jul 10 21:03:05.687429 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.687409 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn" Jul 10 21:03:05.854077 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.854050 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7plcl"] Jul 10 21:03:05.856833 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:03:05.856808 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ac19b08_450e_40f9_85b0_48e5a3e39174.slice/crio-03fb0b208c0dd117328b0dfd8bdfbbd1f51b277a8a7bcf17308f3aaaabb9b3e4 WatchSource:0}: Error finding container 03fb0b208c0dd117328b0dfd8bdfbbd1f51b277a8a7bcf17308f3aaaabb9b3e4: Status 404 returned error can't find the container with id 03fb0b208c0dd117328b0dfd8bdfbbd1f51b277a8a7bcf17308f3aaaabb9b3e4 Jul 10 21:03:05.872180 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:05.872151 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn"] Jul 10 21:03:05.874741 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:03:05.874717 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod50226488_7e48_4dc6_b874_0f373bd4200f.slice/crio-9ac58d3cfb9c5de6dbfc4b29814526459262da2905e63a0a46ac8e585e6221aa WatchSource:0}: Error finding container 9ac58d3cfb9c5de6dbfc4b29814526459262da2905e63a0a46ac8e585e6221aa: Status 404 returned error can't find the container with id 9ac58d3cfb9c5de6dbfc4b29814526459262da2905e63a0a46ac8e585e6221aa Jul 10 21:03:06.245258 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:06.245227 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn" event={"ID":"50226488-7e48-4dc6-b874-0f373bd4200f","Type":"ContainerStarted","Data":"9ac58d3cfb9c5de6dbfc4b29814526459262da2905e63a0a46ac8e585e6221aa"} Jul 10 21:03:06.246269 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:06.246243 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7plcl" event={"ID":"8ac19b08-450e-40f9-85b0-48e5a3e39174","Type":"ContainerStarted","Data":"03fb0b208c0dd117328b0dfd8bdfbbd1f51b277a8a7bcf17308f3aaaabb9b3e4"} Jul 10 21:03:08.612379 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:08.612352 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qftrp"] Jul 10 21:03:08.612691 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:08.612581 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" podUID="8f0b4a83-ed9b-4750-9662-9b7472258bd7" containerName="discovery" containerID="cri-o://52db59590f359843daadf63db8400b27e32dedd5baebca07b39cbd638aee33da" gracePeriod=30 Jul 10 21:03:09.260134 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.260106 2566 generic.go:356] "Generic (PLEG): container finished" podID="8f0b4a83-ed9b-4750-9662-9b7472258bd7" containerID="52db59590f359843daadf63db8400b27e32dedd5baebca07b39cbd638aee33da" exitCode=0 Jul 10 21:03:09.260236 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.260184 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" event={"ID":"8f0b4a83-ed9b-4750-9662-9b7472258bd7","Type":"ContainerDied","Data":"52db59590f359843daadf63db8400b27e32dedd5baebca07b39cbd638aee33da"} Jul 10 21:03:09.261758 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.261732 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7plcl" event={"ID":"8ac19b08-450e-40f9-85b0-48e5a3e39174","Type":"ContainerStarted","Data":"f735ee02e414dd7c71dc88c5658c5cb126a148de14102fb48cf79d81107fef76"} Jul 10 21:03:09.261861 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.261845 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7plcl" Jul 10 21:03:09.280741 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.280690 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7plcl" podStartSLOduration=1.547717634 podStartE2EDuration="4.280675308s" totalImagesPullingTime="2.732957674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:03:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:03:09.279083633 +0000 UTC m=+322.862779235" watchObservedRunningTime="2026-07-10 21:03:09.280675308 +0000 UTC m=+322.864370910" Jul 10 21:03:09.390940 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.390909 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:03:09.567356 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.567249 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhr55\" (UniqueName: \"kubernetes.io/projected/8f0b4a83-ed9b-4750-9662-9b7472258bd7-kube-api-access-nhr55\") pod \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " Jul 10 21:03:09.567356 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.567298 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-cacerts\") pod \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " Jul 10 21:03:09.567356 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.567354 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-csr-dns-cert\") pod \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " Jul 10 21:03:09.567864 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.567385 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-kubeconfig\") pod \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " Jul 10 21:03:09.567864 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.567429 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0b4a83-ed9b-4750-9662-9b7472258bd7-local-certs\") pod \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " Jul 10 21:03:09.567864 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.567467 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-token\") pod \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " Jul 10 21:03:09.567864 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.567506 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-csr-ca-configmap\") pod \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\" (UID: \"8f0b4a83-ed9b-4750-9662-9b7472258bd7\") " Jul 10 21:03:09.568076 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.567911 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "8f0b4a83-ed9b-4750-9662-9b7472258bd7" (UID: "8f0b4a83-ed9b-4750-9662-9b7472258bd7"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:03:09.569906 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.569868 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-cacerts" (OuterVolumeSpecName: "cacerts") pod "8f0b4a83-ed9b-4750-9662-9b7472258bd7" (UID: "8f0b4a83-ed9b-4750-9662-9b7472258bd7"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 21:03:09.570018 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.569920 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f0b4a83-ed9b-4750-9662-9b7472258bd7-kube-api-access-nhr55" (OuterVolumeSpecName: "kube-api-access-nhr55") pod "8f0b4a83-ed9b-4750-9662-9b7472258bd7" (UID: "8f0b4a83-ed9b-4750-9662-9b7472258bd7"). InnerVolumeSpecName "kube-api-access-nhr55". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:03:09.570018 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.570006 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "8f0b4a83-ed9b-4750-9662-9b7472258bd7" (UID: "8f0b4a83-ed9b-4750-9662-9b7472258bd7"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 21:03:09.570138 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.570077 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "8f0b4a83-ed9b-4750-9662-9b7472258bd7" (UID: "8f0b4a83-ed9b-4750-9662-9b7472258bd7"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 21:03:09.570138 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.570120 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f0b4a83-ed9b-4750-9662-9b7472258bd7" (UID: "8f0b4a83-ed9b-4750-9662-9b7472258bd7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:03:09.570213 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.570194 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0b4a83-ed9b-4750-9662-9b7472258bd7-local-certs" (OuterVolumeSpecName: "local-certs") pod "8f0b4a83-ed9b-4750-9662-9b7472258bd7" (UID: "8f0b4a83-ed9b-4750-9662-9b7472258bd7"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:03:09.668823 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.668787 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-token\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:03:09.668823 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.668817 2566 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-csr-ca-configmap\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:03:09.668823 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.668827 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhr55\" (UniqueName: \"kubernetes.io/projected/8f0b4a83-ed9b-4750-9662-9b7472258bd7-kube-api-access-nhr55\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:03:09.669250 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.668837 2566 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-cacerts\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:03:09.669250 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.668847 2566 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-csr-dns-cert\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:03:09.669250 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.668856 2566 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8f0b4a83-ed9b-4750-9662-9b7472258bd7-istio-kubeconfig\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:03:09.669250 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:09.668864 2566 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0b4a83-ed9b-4750-9662-9b7472258bd7-local-certs\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:03:10.267542 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:10.267512 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" Jul 10 21:03:10.267748 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:10.267514 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qftrp" event={"ID":"8f0b4a83-ed9b-4750-9662-9b7472258bd7","Type":"ContainerDied","Data":"6d81993a078e724988e374f8e513c75e068ec9b081446b767927422cd2ceaccd"} Jul 10 21:03:10.267748 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:10.267674 2566 scope.go:117] "RemoveContainer" containerID="52db59590f359843daadf63db8400b27e32dedd5baebca07b39cbd638aee33da" Jul 10 21:03:10.269302 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:10.269274 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn" event={"ID":"50226488-7e48-4dc6-b874-0f373bd4200f","Type":"ContainerStarted","Data":"a57584e4be254763719b9fee0a86c1e19cc3b585f88635995be73368086b7323"} Jul 10 21:03:10.269512 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:10.269434 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn" Jul 10 21:03:10.302416 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:10.302332 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn" podStartSLOduration=1.8693127619999999 podStartE2EDuration="5.302312114s" totalImagesPullingTime="3.432999352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:03:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:03:10.301493893 +0000 UTC m=+323.885189496" watchObservedRunningTime="2026-07-10 21:03:10.302312114 +0000 UTC m=+323.886007716" Jul 10 21:03:10.323786 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:10.323752 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qftrp"] Jul 10 21:03:10.332713 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:10.332688 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qftrp"] Jul 10 21:03:11.067310 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:11.067267 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f0b4a83-ed9b-4750-9662-9b7472258bd7" path="/var/lib/kubelet/pods/8f0b4a83-ed9b-4750-9662-9b7472258bd7/volumes" Jul 10 21:03:12.222010 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:12.221984 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-j5pjd" Jul 10 21:03:12.223864 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:12.223844 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-6qmtk" Jul 10 21:03:40.274999 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:40.274924 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7plcl" Jul 10 21:03:41.275544 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:41.275513 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-22bpn" Jul 10 21:03:54.641029 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:54.640991 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ks6vg"] Jul 10 21:03:54.641471 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:54.641262 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0b4a83-ed9b-4750-9662-9b7472258bd7" containerName="discovery" Jul 10 21:03:54.641471 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:54.641272 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0b4a83-ed9b-4750-9662-9b7472258bd7" containerName="discovery" Jul 10 21:03:54.641471 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:54.641332 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0b4a83-ed9b-4750-9662-9b7472258bd7" containerName="discovery" Jul 10 21:03:54.646027 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:54.645992 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ks6vg" Jul 10 21:03:54.648491 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:54.648465 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 21:03:54.649391 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:54.649368 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-hx8dd\"" Jul 10 21:03:54.649804 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:54.649424 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 21:03:54.655098 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:54.655070 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ks6vg"] Jul 10 21:03:54.688846 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:54.688813 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9nk6\" (UniqueName: \"kubernetes.io/projected/b14b1b21-1585-46ea-bc3a-78fa03ab7217-kube-api-access-m9nk6\") pod \"kuadrant-operator-catalog-ks6vg\" (UID: \"b14b1b21-1585-46ea-bc3a-78fa03ab7217\") " pod="kuadrant-system/kuadrant-operator-catalog-ks6vg" Jul 10 21:03:54.789903 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:54.789861 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9nk6\" (UniqueName: \"kubernetes.io/projected/b14b1b21-1585-46ea-bc3a-78fa03ab7217-kube-api-access-m9nk6\") pod \"kuadrant-operator-catalog-ks6vg\" (UID: \"b14b1b21-1585-46ea-bc3a-78fa03ab7217\") " pod="kuadrant-system/kuadrant-operator-catalog-ks6vg" Jul 10 21:03:54.797681 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:54.797649 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9nk6\" (UniqueName: \"kubernetes.io/projected/b14b1b21-1585-46ea-bc3a-78fa03ab7217-kube-api-access-m9nk6\") pod \"kuadrant-operator-catalog-ks6vg\" (UID: \"b14b1b21-1585-46ea-bc3a-78fa03ab7217\") " pod="kuadrant-system/kuadrant-operator-catalog-ks6vg" Jul 10 21:03:54.958266 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:54.958127 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ks6vg" Jul 10 21:03:55.000300 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:55.000263 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ks6vg"] Jul 10 21:03:55.088754 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:55.088722 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ks6vg"] Jul 10 21:03:55.092517 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:03:55.092478 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb14b1b21_1585_46ea_bc3a_78fa03ab7217.slice/crio-9855e2b26b56275112e700b454ea9dc1455c78d0d81282e29567f5341c074c8e WatchSource:0}: Error finding container 9855e2b26b56275112e700b454ea9dc1455c78d0d81282e29567f5341c074c8e: Status 404 returned error can't find the container with id 9855e2b26b56275112e700b454ea9dc1455c78d0d81282e29567f5341c074c8e Jul 10 21:03:55.208197 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:55.208156 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lbhcl"] Jul 10 21:03:55.211174 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:55.211154 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lbhcl" Jul 10 21:03:55.219875 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:55.219842 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lbhcl"] Jul 10 21:03:55.293978 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:55.293938 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh6gn\" (UniqueName: \"kubernetes.io/projected/714d6812-783b-4e91-a033-04bc634d5062-kube-api-access-vh6gn\") pod \"kuadrant-operator-catalog-lbhcl\" (UID: \"714d6812-783b-4e91-a033-04bc634d5062\") " pod="kuadrant-system/kuadrant-operator-catalog-lbhcl" Jul 10 21:03:55.395320 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:55.395284 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh6gn\" (UniqueName: \"kubernetes.io/projected/714d6812-783b-4e91-a033-04bc634d5062-kube-api-access-vh6gn\") pod \"kuadrant-operator-catalog-lbhcl\" (UID: \"714d6812-783b-4e91-a033-04bc634d5062\") " pod="kuadrant-system/kuadrant-operator-catalog-lbhcl" Jul 10 21:03:55.405341 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:55.405305 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh6gn\" (UniqueName: \"kubernetes.io/projected/714d6812-783b-4e91-a033-04bc634d5062-kube-api-access-vh6gn\") pod \"kuadrant-operator-catalog-lbhcl\" (UID: \"714d6812-783b-4e91-a033-04bc634d5062\") " pod="kuadrant-system/kuadrant-operator-catalog-lbhcl" Jul 10 21:03:55.446424 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:55.446373 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ks6vg" event={"ID":"b14b1b21-1585-46ea-bc3a-78fa03ab7217","Type":"ContainerStarted","Data":"9855e2b26b56275112e700b454ea9dc1455c78d0d81282e29567f5341c074c8e"} Jul 10 21:03:55.520970 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:55.520885 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lbhcl" Jul 10 21:03:55.649312 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:55.649282 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lbhcl"] Jul 10 21:03:55.652211 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:03:55.652183 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod714d6812_783b_4e91_a033_04bc634d5062.slice/crio-c16a2f8d961f765021c4a759203b79e8ec9459be16a31c7a5500b3b21aab1ac7 WatchSource:0}: Error finding container c16a2f8d961f765021c4a759203b79e8ec9459be16a31c7a5500b3b21aab1ac7: Status 404 returned error can't find the container with id c16a2f8d961f765021c4a759203b79e8ec9459be16a31c7a5500b3b21aab1ac7 Jul 10 21:03:56.455604 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:56.455567 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lbhcl" event={"ID":"714d6812-783b-4e91-a033-04bc634d5062","Type":"ContainerStarted","Data":"c16a2f8d961f765021c4a759203b79e8ec9459be16a31c7a5500b3b21aab1ac7"} Jul 10 21:03:58.464143 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:58.464102 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lbhcl" event={"ID":"714d6812-783b-4e91-a033-04bc634d5062","Type":"ContainerStarted","Data":"d80173b3dc9eda0b3e88e05b924d5f0bd6fe669686b20be5defd1dd6092cc295"} Jul 10 21:03:58.465590 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:58.465565 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ks6vg" event={"ID":"b14b1b21-1585-46ea-bc3a-78fa03ab7217","Type":"ContainerStarted","Data":"dbb337c351a8e754fa04564590e372f2d1c00eb998af98e60798a936fdec78eb"} Jul 10 21:03:58.465704 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:58.465591 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-ks6vg" podUID="b14b1b21-1585-46ea-bc3a-78fa03ab7217" containerName="registry-server" containerID="cri-o://dbb337c351a8e754fa04564590e372f2d1c00eb998af98e60798a936fdec78eb" gracePeriod=2 Jul 10 21:03:58.478974 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:58.478928 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-lbhcl" podStartSLOduration=1.6661647670000002 podStartE2EDuration="3.478912788s" totalImagesPullingTime="1.812748021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:03:58.477327235 +0000 UTC m=+372.061022847" watchObservedRunningTime="2026-07-10 21:03:58.478912788 +0000 UTC m=+372.062608390" Jul 10 21:03:58.493709 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:58.493651 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ks6vg" podStartSLOduration=2.118166529 podStartE2EDuration="4.493633467s" totalImagesPullingTime="2.375466938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:03:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:03:58.492021553 +0000 UTC m=+372.075717147" watchObservedRunningTime="2026-07-10 21:03:58.493633467 +0000 UTC m=+372.077329069" Jul 10 21:03:58.708459 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:58.708435 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ks6vg" Jul 10 21:03:58.824476 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:58.824338 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9nk6\" (UniqueName: \"kubernetes.io/projected/b14b1b21-1585-46ea-bc3a-78fa03ab7217-kube-api-access-m9nk6\") pod \"b14b1b21-1585-46ea-bc3a-78fa03ab7217\" (UID: \"b14b1b21-1585-46ea-bc3a-78fa03ab7217\") " Jul 10 21:03:58.826686 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:58.826658 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b14b1b21-1585-46ea-bc3a-78fa03ab7217-kube-api-access-m9nk6" (OuterVolumeSpecName: "kube-api-access-m9nk6") pod "b14b1b21-1585-46ea-bc3a-78fa03ab7217" (UID: "b14b1b21-1585-46ea-bc3a-78fa03ab7217"). InnerVolumeSpecName "kube-api-access-m9nk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:03:58.925313 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:58.925274 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9nk6\" (UniqueName: \"kubernetes.io/projected/b14b1b21-1585-46ea-bc3a-78fa03ab7217-kube-api-access-m9nk6\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:03:59.470674 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:59.470636 2566 generic.go:356] "Generic (PLEG): container finished" podID="b14b1b21-1585-46ea-bc3a-78fa03ab7217" containerID="dbb337c351a8e754fa04564590e372f2d1c00eb998af98e60798a936fdec78eb" exitCode=0 Jul 10 21:03:59.471143 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:59.470681 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ks6vg" Jul 10 21:03:59.471143 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:59.470722 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ks6vg" event={"ID":"b14b1b21-1585-46ea-bc3a-78fa03ab7217","Type":"ContainerDied","Data":"dbb337c351a8e754fa04564590e372f2d1c00eb998af98e60798a936fdec78eb"} Jul 10 21:03:59.471143 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:59.470777 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ks6vg" event={"ID":"b14b1b21-1585-46ea-bc3a-78fa03ab7217","Type":"ContainerDied","Data":"9855e2b26b56275112e700b454ea9dc1455c78d0d81282e29567f5341c074c8e"} Jul 10 21:03:59.471143 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:59.470806 2566 scope.go:117] "RemoveContainer" containerID="dbb337c351a8e754fa04564590e372f2d1c00eb998af98e60798a936fdec78eb" Jul 10 21:03:59.479298 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:59.479278 2566 scope.go:117] "RemoveContainer" containerID="dbb337c351a8e754fa04564590e372f2d1c00eb998af98e60798a936fdec78eb" Jul 10 21:03:59.479627 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:03:59.479605 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbb337c351a8e754fa04564590e372f2d1c00eb998af98e60798a936fdec78eb\": container with ID starting with dbb337c351a8e754fa04564590e372f2d1c00eb998af98e60798a936fdec78eb not found: ID does not exist" containerID="dbb337c351a8e754fa04564590e372f2d1c00eb998af98e60798a936fdec78eb" Jul 10 21:03:59.479698 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:59.479635 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbb337c351a8e754fa04564590e372f2d1c00eb998af98e60798a936fdec78eb"} err="failed to get container status \"dbb337c351a8e754fa04564590e372f2d1c00eb998af98e60798a936fdec78eb\": rpc error: code = NotFound desc = could not find container \"dbb337c351a8e754fa04564590e372f2d1c00eb998af98e60798a936fdec78eb\": container with ID starting with dbb337c351a8e754fa04564590e372f2d1c00eb998af98e60798a936fdec78eb not found: ID does not exist" Jul 10 21:03:59.485148 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:59.485125 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ks6vg"] Jul 10 21:03:59.489078 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:03:59.489054 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ks6vg"] Jul 10 21:04:01.068391 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:01.068356 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b14b1b21-1585-46ea-bc3a-78fa03ab7217" path="/var/lib/kubelet/pods/b14b1b21-1585-46ea-bc3a-78fa03ab7217/volumes" Jul 10 21:04:05.521324 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:05.521286 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-lbhcl" Jul 10 21:04:05.521807 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:05.521568 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-lbhcl" Jul 10 21:04:05.544050 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:05.544021 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-lbhcl" Jul 10 21:04:06.522811 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:06.522785 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-lbhcl" Jul 10 21:04:25.883340 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:25.883300 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4"] Jul 10 21:04:25.883799 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:25.883618 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b14b1b21-1585-46ea-bc3a-78fa03ab7217" containerName="registry-server" Jul 10 21:04:25.883799 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:25.883630 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b14b1b21-1585-46ea-bc3a-78fa03ab7217" containerName="registry-server" Jul 10 21:04:25.883799 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:25.883693 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b14b1b21-1585-46ea-bc3a-78fa03ab7217" containerName="registry-server" Jul 10 21:04:25.886525 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:25.886504 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:25.888786 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:25.888771 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-rs8h7\"" Jul 10 21:04:25.898710 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:25.898685 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4"] Jul 10 21:04:26.048259 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:26.048225 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wfrb\" (UniqueName: \"kubernetes.io/projected/dce43d22-2563-4cc1-8b9e-49bbed4fc908-kube-api-access-5wfrb\") pod \"limitador-operator-controller-manager-859d7b78bd-dzzq4\" (UID: \"dce43d22-2563-4cc1-8b9e-49bbed4fc908\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:26.149489 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:26.149359 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wfrb\" (UniqueName: \"kubernetes.io/projected/dce43d22-2563-4cc1-8b9e-49bbed4fc908-kube-api-access-5wfrb\") pod \"limitador-operator-controller-manager-859d7b78bd-dzzq4\" (UID: \"dce43d22-2563-4cc1-8b9e-49bbed4fc908\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:26.160037 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:26.160003 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wfrb\" (UniqueName: \"kubernetes.io/projected/dce43d22-2563-4cc1-8b9e-49bbed4fc908-kube-api-access-5wfrb\") pod \"limitador-operator-controller-manager-859d7b78bd-dzzq4\" (UID: \"dce43d22-2563-4cc1-8b9e-49bbed4fc908\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:26.195982 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:26.195944 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:26.330763 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:26.330739 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4"] Jul 10 21:04:26.333549 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:04:26.333514 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddce43d22_2563_4cc1_8b9e_49bbed4fc908.slice/crio-c4bb3782bbfdff75717ee4cb8a31442e846eb54a0c12c25b70292399edf5dab9 WatchSource:0}: Error finding container c4bb3782bbfdff75717ee4cb8a31442e846eb54a0c12c25b70292399edf5dab9: Status 404 returned error can't find the container with id c4bb3782bbfdff75717ee4cb8a31442e846eb54a0c12c25b70292399edf5dab9 Jul 10 21:04:26.582299 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:26.582264 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" event={"ID":"dce43d22-2563-4cc1-8b9e-49bbed4fc908","Type":"ContainerStarted","Data":"c4bb3782bbfdff75717ee4cb8a31442e846eb54a0c12c25b70292399edf5dab9"} Jul 10 21:04:28.592703 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:28.592670 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" event={"ID":"dce43d22-2563-4cc1-8b9e-49bbed4fc908","Type":"ContainerStarted","Data":"d91670d67c2be9de663b9c7b6146e49c19b0d94197f29d9c9631fdbcef80ff9d"} Jul 10 21:04:28.593111 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:28.592804 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:28.607747 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:28.607678 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" podStartSLOduration=1.576439609 podStartE2EDuration="3.607661375s" totalImagesPullingTime="2.031221766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:04:28.607625682 +0000 UTC m=+402.191321288" watchObservedRunningTime="2026-07-10 21:04:28.607661375 +0000 UTC m=+402.191356972" Jul 10 21:04:29.343626 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:29.343573 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-z5fp6"] Jul 10 21:04:29.346790 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:29.346768 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z5fp6" Jul 10 21:04:29.349277 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:29.349254 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-vf2b7\"" Jul 10 21:04:29.349277 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:29.349272 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 10 21:04:29.356764 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:29.356740 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-z5fp6"] Jul 10 21:04:29.475992 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:29.475953 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj4r6\" (UniqueName: \"kubernetes.io/projected/0eef96a5-6001-44b5-b5a8-d38f66be3f42-kube-api-access-gj4r6\") pod \"dns-operator-controller-manager-864dd78c4c-z5fp6\" (UID: \"0eef96a5-6001-44b5-b5a8-d38f66be3f42\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z5fp6" Jul 10 21:04:29.577125 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:29.577083 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj4r6\" (UniqueName: \"kubernetes.io/projected/0eef96a5-6001-44b5-b5a8-d38f66be3f42-kube-api-access-gj4r6\") pod \"dns-operator-controller-manager-864dd78c4c-z5fp6\" (UID: \"0eef96a5-6001-44b5-b5a8-d38f66be3f42\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z5fp6" Jul 10 21:04:29.588218 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:29.588189 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj4r6\" (UniqueName: \"kubernetes.io/projected/0eef96a5-6001-44b5-b5a8-d38f66be3f42-kube-api-access-gj4r6\") pod \"dns-operator-controller-manager-864dd78c4c-z5fp6\" (UID: \"0eef96a5-6001-44b5-b5a8-d38f66be3f42\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z5fp6" Jul 10 21:04:29.658014 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:29.657908 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z5fp6" Jul 10 21:04:29.792830 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:29.792798 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-z5fp6"] Jul 10 21:04:29.796128 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:04:29.796099 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eef96a5_6001_44b5_b5a8_d38f66be3f42.slice/crio-3f3600395bbecbfe737b250ac75c532f3041d0c6d36350cd1f88af69cbd52894 WatchSource:0}: Error finding container 3f3600395bbecbfe737b250ac75c532f3041d0c6d36350cd1f88af69cbd52894: Status 404 returned error can't find the container with id 3f3600395bbecbfe737b250ac75c532f3041d0c6d36350cd1f88af69cbd52894 Jul 10 21:04:30.602116 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:30.602083 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z5fp6" event={"ID":"0eef96a5-6001-44b5-b5a8-d38f66be3f42","Type":"ContainerStarted","Data":"3f3600395bbecbfe737b250ac75c532f3041d0c6d36350cd1f88af69cbd52894"} Jul 10 21:04:32.612860 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:32.612828 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z5fp6" event={"ID":"0eef96a5-6001-44b5-b5a8-d38f66be3f42","Type":"ContainerStarted","Data":"e2a284366a8b4cd338a4b6601b3cfa892c5678e50f4832121ff2aedda2da6c75"} Jul 10 21:04:32.613298 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:32.612908 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z5fp6" Jul 10 21:04:32.633922 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:32.633872 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z5fp6" podStartSLOduration=1.581730716 podStartE2EDuration="3.633856263s" totalImagesPullingTime="2.052125547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:04:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:04:32.632319027 +0000 UTC m=+406.216014631" watchObservedRunningTime="2026-07-10 21:04:32.633856263 +0000 UTC m=+406.217551958" Jul 10 21:04:33.094269 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:33.094232 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g78jw"] Jul 10 21:04:33.097586 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:33.097564 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g78jw" Jul 10 21:04:33.100712 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:33.100689 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-xfp7c\"" Jul 10 21:04:33.118776 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:33.118749 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g78jw"] Jul 10 21:04:33.208383 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:33.208340 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw7mg\" (UniqueName: \"kubernetes.io/projected/3d49ffc9-3b56-41eb-b6ad-9ef2d49f7d75-kube-api-access-kw7mg\") pod \"authorino-operator-75cc8d5cb7-g78jw\" (UID: \"3d49ffc9-3b56-41eb-b6ad-9ef2d49f7d75\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g78jw" Jul 10 21:04:33.309070 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:33.309037 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw7mg\" (UniqueName: \"kubernetes.io/projected/3d49ffc9-3b56-41eb-b6ad-9ef2d49f7d75-kube-api-access-kw7mg\") pod \"authorino-operator-75cc8d5cb7-g78jw\" (UID: \"3d49ffc9-3b56-41eb-b6ad-9ef2d49f7d75\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g78jw" Jul 10 21:04:33.317304 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:33.317274 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw7mg\" (UniqueName: \"kubernetes.io/projected/3d49ffc9-3b56-41eb-b6ad-9ef2d49f7d75-kube-api-access-kw7mg\") pod \"authorino-operator-75cc8d5cb7-g78jw\" (UID: \"3d49ffc9-3b56-41eb-b6ad-9ef2d49f7d75\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g78jw" Jul 10 21:04:33.407904 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:33.407803 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g78jw" Jul 10 21:04:33.544446 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:33.544420 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g78jw"] Jul 10 21:04:33.546487 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:04:33.546462 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d49ffc9_3b56_41eb_b6ad_9ef2d49f7d75.slice/crio-1f9f5070231b32aaf239ed99726f13bb9e8193e18d62c7cfbb5acd888fc10198 WatchSource:0}: Error finding container 1f9f5070231b32aaf239ed99726f13bb9e8193e18d62c7cfbb5acd888fc10198: Status 404 returned error can't find the container with id 1f9f5070231b32aaf239ed99726f13bb9e8193e18d62c7cfbb5acd888fc10198 Jul 10 21:04:33.619334 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:33.619303 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g78jw" event={"ID":"3d49ffc9-3b56-41eb-b6ad-9ef2d49f7d75","Type":"ContainerStarted","Data":"1f9f5070231b32aaf239ed99726f13bb9e8193e18d62c7cfbb5acd888fc10198"} Jul 10 21:04:36.634145 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:36.634057 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g78jw" event={"ID":"3d49ffc9-3b56-41eb-b6ad-9ef2d49f7d75","Type":"ContainerStarted","Data":"888284ef90a4eb3c14c286236cbfea2aff77d017356d6f5dd75f2b14950cc419"} Jul 10 21:04:36.634546 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:36.634184 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g78jw" Jul 10 21:04:36.653250 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:36.653195 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g78jw" podStartSLOduration=0.938734362 podStartE2EDuration="3.653176104s" totalImagesPullingTime="2.714441742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:04:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:04:36.653058063 +0000 UTC m=+410.236753666" watchObservedRunningTime="2026-07-10 21:04:36.653176104 +0000 UTC m=+410.236871709" Jul 10 21:04:37.746916 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:37.746879 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn"] Jul 10 21:04:37.750083 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:37.750068 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:37.753776 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:37.753754 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-p7gsq\"" Jul 10 21:04:37.768128 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:37.768097 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn"] Jul 10 21:04:37.840831 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:37.840795 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltm59\" (UniqueName: \"kubernetes.io/projected/df83e09d-a0e9-49fb-ab37-233a5641a984-kube-api-access-ltm59\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" (UID: \"df83e09d-a0e9-49fb-ab37-233a5641a984\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:37.840997 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:37.840864 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df83e09d-a0e9-49fb-ab37-233a5641a984-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" (UID: \"df83e09d-a0e9-49fb-ab37-233a5641a984\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:37.941438 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:37.941407 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltm59\" (UniqueName: \"kubernetes.io/projected/df83e09d-a0e9-49fb-ab37-233a5641a984-kube-api-access-ltm59\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" (UID: \"df83e09d-a0e9-49fb-ab37-233a5641a984\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:37.941565 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:37.941451 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df83e09d-a0e9-49fb-ab37-233a5641a984-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" (UID: \"df83e09d-a0e9-49fb-ab37-233a5641a984\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:37.941782 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:37.941766 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df83e09d-a0e9-49fb-ab37-233a5641a984-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" (UID: \"df83e09d-a0e9-49fb-ab37-233a5641a984\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:37.950849 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:37.950831 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltm59\" (UniqueName: \"kubernetes.io/projected/df83e09d-a0e9-49fb-ab37-233a5641a984-kube-api-access-ltm59\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" (UID: \"df83e09d-a0e9-49fb-ab37-233a5641a984\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:38.059727 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:38.059637 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:38.214337 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:38.214219 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn"] Jul 10 21:04:38.218461 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:04:38.218434 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf83e09d_a0e9_49fb_ab37_233a5641a984.slice/crio-bd500815ba587c880bc713a3e8634deefb4189ecd144ff7e35ad6d2ab77aa130 WatchSource:0}: Error finding container bd500815ba587c880bc713a3e8634deefb4189ecd144ff7e35ad6d2ab77aa130: Status 404 returned error can't find the container with id bd500815ba587c880bc713a3e8634deefb4189ecd144ff7e35ad6d2ab77aa130 Jul 10 21:04:38.644095 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:38.644061 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" event={"ID":"df83e09d-a0e9-49fb-ab37-233a5641a984","Type":"ContainerStarted","Data":"bd500815ba587c880bc713a3e8634deefb4189ecd144ff7e35ad6d2ab77aa130"} Jul 10 21:04:39.598628 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:39.598588 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:43.622339 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:43.622308 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z5fp6" Jul 10 21:04:45.676006 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:45.675975 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" event={"ID":"df83e09d-a0e9-49fb-ab37-233a5641a984","Type":"ContainerStarted","Data":"425c06398e265b1a75c220c51fd055330c8708bb7d0237dff6b26f9434f6dd8d"} Jul 10 21:04:45.676371 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:45.676046 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:45.697219 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:45.697170 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" podStartSLOduration=1.9977402 podStartE2EDuration="8.697154539s" totalImagesPullingTime="6.699414339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:04:45.695319376 +0000 UTC m=+419.279014977" watchObservedRunningTime="2026-07-10 21:04:45.697154539 +0000 UTC m=+419.280850140" Jul 10 21:04:47.640659 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:47.640630 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g78jw" Jul 10 21:04:56.682747 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:56.682715 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:57.855290 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:57.855250 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn"] Jul 10 21:04:57.859897 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:57.859881 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:57.878964 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:57.878935 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn"] Jul 10 21:04:57.896036 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:57.896007 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fedea228-efc3-4da0-9a19-fa56b6c84818-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-spbvn\" (UID: \"fedea228-efc3-4da0-9a19-fa56b6c84818\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:57.896146 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:57.896082 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpztf\" (UniqueName: \"kubernetes.io/projected/fedea228-efc3-4da0-9a19-fa56b6c84818-kube-api-access-bpztf\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-spbvn\" (UID: \"fedea228-efc3-4da0-9a19-fa56b6c84818\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:57.985599 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:57.985556 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn"] Jul 10 21:04:57.985805 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:57.985789 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-bpztf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" podUID="fedea228-efc3-4da0-9a19-fa56b6c84818" Jul 10 21:04:57.996387 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:57.996359 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fedea228-efc3-4da0-9a19-fa56b6c84818-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-spbvn\" (UID: \"fedea228-efc3-4da0-9a19-fa56b6c84818\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:57.996521 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:57.996434 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpztf\" (UniqueName: \"kubernetes.io/projected/fedea228-efc3-4da0-9a19-fa56b6c84818-kube-api-access-bpztf\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-spbvn\" (UID: \"fedea228-efc3-4da0-9a19-fa56b6c84818\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:57.996718 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:57.996698 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fedea228-efc3-4da0-9a19-fa56b6c84818-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-spbvn\" (UID: \"fedea228-efc3-4da0-9a19-fa56b6c84818\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:58.015344 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.015308 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpztf\" (UniqueName: \"kubernetes.io/projected/fedea228-efc3-4da0-9a19-fa56b6c84818-kube-api-access-bpztf\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-spbvn\" (UID: \"fedea228-efc3-4da0-9a19-fa56b6c84818\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:58.024859 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.024832 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl"] Jul 10 21:04:58.027970 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.027955 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.046083 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.046058 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl"] Jul 10 21:04:58.097637 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.097601 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzdd7\" (UniqueName: \"kubernetes.io/projected/b027002c-fd25-4009-8831-a752553a516a-kube-api-access-rzdd7\") pod \"kuadrant-operator-controller-manager-55467d9f7-5wktl\" (UID: \"b027002c-fd25-4009-8831-a752553a516a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.097812 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.097713 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b027002c-fd25-4009-8831-a752553a516a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5wktl\" (UID: \"b027002c-fd25-4009-8831-a752553a516a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.199156 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.199059 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b027002c-fd25-4009-8831-a752553a516a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5wktl\" (UID: \"b027002c-fd25-4009-8831-a752553a516a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.199156 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.199122 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzdd7\" (UniqueName: \"kubernetes.io/projected/b027002c-fd25-4009-8831-a752553a516a-kube-api-access-rzdd7\") pod \"kuadrant-operator-controller-manager-55467d9f7-5wktl\" (UID: \"b027002c-fd25-4009-8831-a752553a516a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.199537 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.199517 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b027002c-fd25-4009-8831-a752553a516a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5wktl\" (UID: \"b027002c-fd25-4009-8831-a752553a516a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.209361 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.209336 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzdd7\" (UniqueName: \"kubernetes.io/projected/b027002c-fd25-4009-8831-a752553a516a-kube-api-access-rzdd7\") pod \"kuadrant-operator-controller-manager-55467d9f7-5wktl\" (UID: \"b027002c-fd25-4009-8831-a752553a516a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.338317 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.338275 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.407204 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.407173 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl"] Jul 10 21:04:58.423281 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.423038 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl"] Jul 10 21:04:58.432805 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.432771 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc"] Jul 10 21:04:58.437873 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.437849 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn"] Jul 10 21:04:58.438050 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.437943 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" Jul 10 21:04:58.449883 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.449814 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn"] Jul 10 21:04:58.451435 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.451356 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc"] Jul 10 21:04:58.466629 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.466578 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn"] Jul 10 21:04:58.467056 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.466999 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" containerName="manager" containerID="cri-o://425c06398e265b1a75c220c51fd055330c8708bb7d0237dff6b26f9434f6dd8d" gracePeriod=2 Jul 10 21:04:58.472716 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.472641 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn"] Jul 10 21:04:58.476287 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.475377 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4"] Jul 10 21:04:58.476287 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.475665 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" podUID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" containerName="manager" containerID="cri-o://d91670d67c2be9de663b9c7b6146e49c19b0d94197f29d9c9631fdbcef80ff9d" gracePeriod=2 Jul 10 21:04:58.477388 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.477345 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:58.479587 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.479548 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:58.484461 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.484388 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4"] Jul 10 21:04:58.491357 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.491327 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m"] Jul 10 21:04:58.491916 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.491658 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" containerName="manager" Jul 10 21:04:58.491916 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.491670 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" containerName="manager" Jul 10 21:04:58.491916 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.491683 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" containerName="manager" Jul 10 21:04:58.491916 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.491688 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" containerName="manager" Jul 10 21:04:58.491916 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.491737 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" containerName="manager" Jul 10 21:04:58.491916 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.491744 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" containerName="manager" Jul 10 21:04:58.495373 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.495356 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" Jul 10 21:04:58.499218 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.499183 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:58.502031 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.502005 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7lzn"] Jul 10 21:04:58.502211 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.502189 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4cd291af-f4d8-400b-9731-76c483d7f1e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-ln2sc\" (UID: \"4cd291af-f4d8-400b-9731-76c483d7f1e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" Jul 10 21:04:58.502308 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.502229 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8qcz\" (UniqueName: \"kubernetes.io/projected/4cd291af-f4d8-400b-9731-76c483d7f1e7-kube-api-access-v8qcz\") pod \"kuadrant-operator-controller-manager-55467d9f7-ln2sc\" (UID: \"4cd291af-f4d8-400b-9731-76c483d7f1e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" Jul 10 21:04:58.503635 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.503611 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dzzq4\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:58.506365 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.506342 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7lzn" Jul 10 21:04:58.506845 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.506824 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m"] Jul 10 21:04:58.528517 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.528477 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7lzn"] Jul 10 21:04:58.532556 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.532518 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:58.534267 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.534234 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dzzq4\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:58.603637 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.603598 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d008e27f-58f3-4294-9f36-1976ba653620-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rsr8m\" (UID: \"d008e27f-58f3-4294-9f36-1976ba653620\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" Jul 10 21:04:58.603786 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.603666 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nfks\" (UniqueName: \"kubernetes.io/projected/32440622-a23a-4e8a-9cca-666f88134ee6-kube-api-access-7nfks\") pod \"limitador-operator-controller-manager-859d7b78bd-j7lzn\" (UID: \"32440622-a23a-4e8a-9cca-666f88134ee6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7lzn" Jul 10 21:04:58.603786 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.603691 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4cd291af-f4d8-400b-9731-76c483d7f1e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-ln2sc\" (UID: \"4cd291af-f4d8-400b-9731-76c483d7f1e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" Jul 10 21:04:58.603786 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.603722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxfqn\" (UniqueName: \"kubernetes.io/projected/d008e27f-58f3-4294-9f36-1976ba653620-kube-api-access-qxfqn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rsr8m\" (UID: \"d008e27f-58f3-4294-9f36-1976ba653620\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" Jul 10 21:04:58.603786 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.603745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8qcz\" (UniqueName: \"kubernetes.io/projected/4cd291af-f4d8-400b-9731-76c483d7f1e7-kube-api-access-v8qcz\") pod \"kuadrant-operator-controller-manager-55467d9f7-ln2sc\" (UID: \"4cd291af-f4d8-400b-9731-76c483d7f1e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" Jul 10 21:04:58.604424 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.604368 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4cd291af-f4d8-400b-9731-76c483d7f1e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-ln2sc\" (UID: \"4cd291af-f4d8-400b-9731-76c483d7f1e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" Jul 10 21:04:58.614719 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.614667 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8qcz\" (UniqueName: \"kubernetes.io/projected/4cd291af-f4d8-400b-9731-76c483d7f1e7-kube-api-access-v8qcz\") pod \"kuadrant-operator-controller-manager-55467d9f7-ln2sc\" (UID: \"4cd291af-f4d8-400b-9731-76c483d7f1e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" Jul 10 21:04:58.651852 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.651820 2566 log.go:32] "RunPodSandbox from runtime service failed" err=< Jul 10 21:04:58.651852 ip-10-0-131-227 kubenswrapper[2566]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-5wktl_kuadrant-system_b027002c-fd25-4009-8831-a752553a516a_0(1628d6212e75aa32dd515a7500d5086b70229d92e239bced6ec9698a9b65aad6): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-5wktl to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"1628d6212e75aa32dd515a7500d5086b70229d92e239bced6ec9698a9b65aad6" Netns:"/var/run/netns/29c9e2a5-fed2-44a7-99a2-8e10b39297c2" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-5wktl;K8S_POD_INFRA_CONTAINER_ID=1628d6212e75aa32dd515a7500d5086b70229d92e239bced6ec9698a9b65aad6;K8S_POD_UID=b027002c-fd25-4009-8831-a752553a516a" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl/b027002c-fd25-4009-8831-a752553a516a:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl 1628d6212e75aa32dd515a7500d5086b70229d92e239bced6ec9698a9b65aad6 network default NAD default NAD key default] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl 1628d6212e75aa32dd515a7500d5086b70229d92e239bced6ec9698a9b65aad6 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:86:00:1b [10.134.0.27/23] Jul 10 21:04:58.651852 ip-10-0-131-227 kubenswrapper[2566]: ' Jul 10 21:04:58.651852 ip-10-0-131-227 kubenswrapper[2566]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 10 21:04:58.651852 ip-10-0-131-227 kubenswrapper[2566]: > Jul 10 21:04:58.652149 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.651878 2566 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=< Jul 10 21:04:58.652149 ip-10-0-131-227 kubenswrapper[2566]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-5wktl_kuadrant-system_b027002c-fd25-4009-8831-a752553a516a_0(1628d6212e75aa32dd515a7500d5086b70229d92e239bced6ec9698a9b65aad6): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-5wktl to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"1628d6212e75aa32dd515a7500d5086b70229d92e239bced6ec9698a9b65aad6" Netns:"/var/run/netns/29c9e2a5-fed2-44a7-99a2-8e10b39297c2" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-5wktl;K8S_POD_INFRA_CONTAINER_ID=1628d6212e75aa32dd515a7500d5086b70229d92e239bced6ec9698a9b65aad6;K8S_POD_UID=b027002c-fd25-4009-8831-a752553a516a" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl/b027002c-fd25-4009-8831-a752553a516a:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl 1628d6212e75aa32dd515a7500d5086b70229d92e239bced6ec9698a9b65aad6 network default NAD default NAD key default] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl 1628d6212e75aa32dd515a7500d5086b70229d92e239bced6ec9698a9b65aad6 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:86:00:1b [10.134.0.27/23] Jul 10 21:04:58.652149 ip-10-0-131-227 kubenswrapper[2566]: ' Jul 10 21:04:58.652149 ip-10-0-131-227 kubenswrapper[2566]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 10 21:04:58.652149 ip-10-0-131-227 kubenswrapper[2566]: > pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.705137 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.705038 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nfks\" (UniqueName: \"kubernetes.io/projected/32440622-a23a-4e8a-9cca-666f88134ee6-kube-api-access-7nfks\") pod \"limitador-operator-controller-manager-859d7b78bd-j7lzn\" (UID: \"32440622-a23a-4e8a-9cca-666f88134ee6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7lzn" Jul 10 21:04:58.705137 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.705088 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxfqn\" (UniqueName: \"kubernetes.io/projected/d008e27f-58f3-4294-9f36-1976ba653620-kube-api-access-qxfqn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rsr8m\" (UID: \"d008e27f-58f3-4294-9f36-1976ba653620\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" Jul 10 21:04:58.705137 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.705128 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d008e27f-58f3-4294-9f36-1976ba653620-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rsr8m\" (UID: \"d008e27f-58f3-4294-9f36-1976ba653620\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" Jul 10 21:04:58.705509 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.705493 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d008e27f-58f3-4294-9f36-1976ba653620-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rsr8m\" (UID: \"d008e27f-58f3-4294-9f36-1976ba653620\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" Jul 10 21:04:58.714883 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.714846 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxfqn\" (UniqueName: \"kubernetes.io/projected/d008e27f-58f3-4294-9f36-1976ba653620-kube-api-access-qxfqn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rsr8m\" (UID: \"d008e27f-58f3-4294-9f36-1976ba653620\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" Jul 10 21:04:58.715036 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.714966 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nfks\" (UniqueName: \"kubernetes.io/projected/32440622-a23a-4e8a-9cca-666f88134ee6-kube-api-access-7nfks\") pod \"limitador-operator-controller-manager-859d7b78bd-j7lzn\" (UID: \"32440622-a23a-4e8a-9cca-666f88134ee6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7lzn" Jul 10 21:04:58.725552 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.725522 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:58.727999 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.727967 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:58.729983 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.729959 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dzzq4\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:58.730208 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.730189 2566 generic.go:356] "Generic (PLEG): container finished" podID="df83e09d-a0e9-49fb-ab37-233a5641a984" containerID="425c06398e265b1a75c220c51fd055330c8708bb7d0237dff6b26f9434f6dd8d" exitCode=0 Jul 10 21:04:58.730283 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.730243 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:58.730333 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.730295 2566 scope.go:117] "RemoveContainer" containerID="425c06398e265b1a75c220c51fd055330c8708bb7d0237dff6b26f9434f6dd8d" Jul 10 21:04:58.732083 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.732061 2566 generic.go:356] "Generic (PLEG): container finished" podID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" containerID="d91670d67c2be9de663b9c7b6146e49c19b0d94197f29d9c9631fdbcef80ff9d" exitCode=0 Jul 10 21:04:58.732180 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.732148 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:58.732180 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.732154 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.732491 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.732464 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:58.734206 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.734177 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dzzq4\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:58.735897 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.735865 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:58.737531 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.737500 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dzzq4\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:58.739443 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.739419 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-spbvn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="fedea228-efc3-4da0-9a19-fa56b6c84818" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:58.739560 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.739528 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:58.739687 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.739628 2566 scope.go:117] "RemoveContainer" containerID="425c06398e265b1a75c220c51fd055330c8708bb7d0237dff6b26f9434f6dd8d" Jul 10 21:04:58.739923 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.739898 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"425c06398e265b1a75c220c51fd055330c8708bb7d0237dff6b26f9434f6dd8d\": container with ID starting with 425c06398e265b1a75c220c51fd055330c8708bb7d0237dff6b26f9434f6dd8d not found: ID does not exist" containerID="425c06398e265b1a75c220c51fd055330c8708bb7d0237dff6b26f9434f6dd8d" Jul 10 21:04:58.739976 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.739935 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"425c06398e265b1a75c220c51fd055330c8708bb7d0237dff6b26f9434f6dd8d"} err="failed to get container status \"425c06398e265b1a75c220c51fd055330c8708bb7d0237dff6b26f9434f6dd8d\": rpc error: code = NotFound desc = could not find container \"425c06398e265b1a75c220c51fd055330c8708bb7d0237dff6b26f9434f6dd8d\": container with ID starting with 425c06398e265b1a75c220c51fd055330c8708bb7d0237dff6b26f9434f6dd8d not found: ID does not exist" Jul 10 21:04:58.741246 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.741226 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5wktl\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="b027002c-fd25-4009-8831-a752553a516a" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.742976 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.742953 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:58.744242 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.744223 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.744682 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.744662 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dzzq4\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:58.746278 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.746256 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-spbvn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="fedea228-efc3-4da0-9a19-fa56b6c84818" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:58.747944 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.747923 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5wktl\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="b027002c-fd25-4009-8831-a752553a516a" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.749818 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.749791 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:58.750929 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.750909 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" Jul 10 21:04:58.752099 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.752073 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dzzq4\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:58.753797 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.753719 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-spbvn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="fedea228-efc3-4da0-9a19-fa56b6c84818" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:58.755418 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.755381 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5wktl\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="b027002c-fd25-4009-8831-a752553a516a" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.761502 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.761479 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:58.763549 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.763520 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-spbvn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="fedea228-efc3-4da0-9a19-fa56b6c84818" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:58.765097 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.765074 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5wktl\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="b027002c-fd25-4009-8831-a752553a516a" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:58.766940 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.766908 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:58.768688 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:58.768670 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dzzq4\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:58.807996 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.806175 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b027002c-fd25-4009-8831-a752553a516a-extensions-socket-volume\") pod \"b027002c-fd25-4009-8831-a752553a516a\" (UID: \"b027002c-fd25-4009-8831-a752553a516a\") " Jul 10 21:04:58.807996 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.806250 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpztf\" (UniqueName: \"kubernetes.io/projected/fedea228-efc3-4da0-9a19-fa56b6c84818-kube-api-access-bpztf\") pod \"fedea228-efc3-4da0-9a19-fa56b6c84818\" (UID: \"fedea228-efc3-4da0-9a19-fa56b6c84818\") " Jul 10 21:04:58.807996 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.806296 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltm59\" (UniqueName: \"kubernetes.io/projected/df83e09d-a0e9-49fb-ab37-233a5641a984-kube-api-access-ltm59\") pod \"df83e09d-a0e9-49fb-ab37-233a5641a984\" (UID: \"df83e09d-a0e9-49fb-ab37-233a5641a984\") " Jul 10 21:04:58.807996 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.806324 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df83e09d-a0e9-49fb-ab37-233a5641a984-extensions-socket-volume\") pod \"df83e09d-a0e9-49fb-ab37-233a5641a984\" (UID: \"df83e09d-a0e9-49fb-ab37-233a5641a984\") " Jul 10 21:04:58.807996 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.806363 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fedea228-efc3-4da0-9a19-fa56b6c84818-extensions-socket-volume\") pod \"fedea228-efc3-4da0-9a19-fa56b6c84818\" (UID: \"fedea228-efc3-4da0-9a19-fa56b6c84818\") " Jul 10 21:04:58.807996 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.806414 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wfrb\" (UniqueName: \"kubernetes.io/projected/dce43d22-2563-4cc1-8b9e-49bbed4fc908-kube-api-access-5wfrb\") pod \"dce43d22-2563-4cc1-8b9e-49bbed4fc908\" (UID: \"dce43d22-2563-4cc1-8b9e-49bbed4fc908\") " Jul 10 21:04:58.807996 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.806470 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzdd7\" (UniqueName: \"kubernetes.io/projected/b027002c-fd25-4009-8831-a752553a516a-kube-api-access-rzdd7\") pod \"b027002c-fd25-4009-8831-a752553a516a\" (UID: \"b027002c-fd25-4009-8831-a752553a516a\") " Jul 10 21:04:58.807996 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.807779 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b027002c-fd25-4009-8831-a752553a516a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b027002c-fd25-4009-8831-a752553a516a" (UID: "b027002c-fd25-4009-8831-a752553a516a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:04:58.808562 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.808420 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fedea228-efc3-4da0-9a19-fa56b6c84818-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fedea228-efc3-4da0-9a19-fa56b6c84818" (UID: "fedea228-efc3-4da0-9a19-fa56b6c84818"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:04:58.808805 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.808625 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df83e09d-a0e9-49fb-ab37-233a5641a984-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "df83e09d-a0e9-49fb-ab37-233a5641a984" (UID: "df83e09d-a0e9-49fb-ab37-233a5641a984"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:04:58.809799 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.809772 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b027002c-fd25-4009-8831-a752553a516a-kube-api-access-rzdd7" (OuterVolumeSpecName: "kube-api-access-rzdd7") pod "b027002c-fd25-4009-8831-a752553a516a" (UID: "b027002c-fd25-4009-8831-a752553a516a"). InnerVolumeSpecName "kube-api-access-rzdd7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:04:58.809934 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.809910 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df83e09d-a0e9-49fb-ab37-233a5641a984-kube-api-access-ltm59" (OuterVolumeSpecName: "kube-api-access-ltm59") pod "df83e09d-a0e9-49fb-ab37-233a5641a984" (UID: "df83e09d-a0e9-49fb-ab37-233a5641a984"). InnerVolumeSpecName "kube-api-access-ltm59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:04:58.810523 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.810502 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dce43d22-2563-4cc1-8b9e-49bbed4fc908-kube-api-access-5wfrb" (OuterVolumeSpecName: "kube-api-access-5wfrb") pod "dce43d22-2563-4cc1-8b9e-49bbed4fc908" (UID: "dce43d22-2563-4cc1-8b9e-49bbed4fc908"). InnerVolumeSpecName "kube-api-access-5wfrb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:04:58.811710 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.811683 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fedea228-efc3-4da0-9a19-fa56b6c84818-kube-api-access-bpztf" (OuterVolumeSpecName: "kube-api-access-bpztf") pod "fedea228-efc3-4da0-9a19-fa56b6c84818" (UID: "fedea228-efc3-4da0-9a19-fa56b6c84818"). InnerVolumeSpecName "kube-api-access-bpztf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:04:58.884797 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.884771 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc"] Jul 10 21:04:58.887438 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:04:58.887378 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cd291af_f4d8_400b_9731_76c483d7f1e7.slice/crio-12daae2b8e7ad481837d0fee48696249a97e87b6c94229d551c74eba5ef7b001 WatchSource:0}: Error finding container 12daae2b8e7ad481837d0fee48696249a97e87b6c94229d551c74eba5ef7b001: Status 404 returned error can't find the container with id 12daae2b8e7ad481837d0fee48696249a97e87b6c94229d551c74eba5ef7b001 Jul 10 21:04:58.895598 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.895580 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" Jul 10 21:04:58.904515 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.904490 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7lzn" Jul 10 21:04:58.907731 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.907707 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b027002c-fd25-4009-8831-a752553a516a-extensions-socket-volume\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:04:58.907791 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.907740 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpztf\" (UniqueName: \"kubernetes.io/projected/fedea228-efc3-4da0-9a19-fa56b6c84818-kube-api-access-bpztf\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:04:58.907791 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.907758 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltm59\" (UniqueName: \"kubernetes.io/projected/df83e09d-a0e9-49fb-ab37-233a5641a984-kube-api-access-ltm59\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:04:58.907948 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.907926 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df83e09d-a0e9-49fb-ab37-233a5641a984-extensions-socket-volume\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:04:58.907993 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.907957 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fedea228-efc3-4da0-9a19-fa56b6c84818-extensions-socket-volume\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:04:58.907993 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.907972 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wfrb\" (UniqueName: \"kubernetes.io/projected/dce43d22-2563-4cc1-8b9e-49bbed4fc908-kube-api-access-5wfrb\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:04:58.907993 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:58.907985 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzdd7\" (UniqueName: \"kubernetes.io/projected/b027002c-fd25-4009-8831-a752553a516a-kube-api-access-rzdd7\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:04:59.048942 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:59.048907 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xt5xn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xt5xn" Jul 10 21:04:59.050771 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:59.050746 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-dzzq4\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:59.053020 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:59.052895 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-spbvn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="fedea228-efc3-4da0-9a19-fa56b6c84818" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:59.054479 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.054457 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m"] Jul 10 21:04:59.054764 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:59.054727 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5wktl\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="b027002c-fd25-4009-8831-a752553a516a" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:59.057558 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:04:59.057533 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd008e27f_58f3_4294_9f36_1976ba653620.slice/crio-c60fa68b087fbaf9bd61c493685ce775a0a22d4654aba42d9f2628dd0e942137 WatchSource:0}: Error finding container c60fa68b087fbaf9bd61c493685ce775a0a22d4654aba42d9f2628dd0e942137: Status 404 returned error can't find the container with id c60fa68b087fbaf9bd61c493685ce775a0a22d4654aba42d9f2628dd0e942137 Jul 10 21:04:59.067790 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.067760 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b027002c-fd25-4009-8831-a752553a516a" path="/var/lib/kubelet/pods/b027002c-fd25-4009-8831-a752553a516a/volumes" Jul 10 21:04:59.068072 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.068054 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dce43d22-2563-4cc1-8b9e-49bbed4fc908" path="/var/lib/kubelet/pods/dce43d22-2563-4cc1-8b9e-49bbed4fc908/volumes" Jul 10 21:04:59.068463 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.068445 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df83e09d-a0e9-49fb-ab37-233a5641a984" path="/var/lib/kubelet/pods/df83e09d-a0e9-49fb-ab37-233a5641a984/volumes" Jul 10 21:04:59.068804 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.068788 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fedea228-efc3-4da0-9a19-fa56b6c84818" path="/var/lib/kubelet/pods/fedea228-efc3-4da0-9a19-fa56b6c84818/volumes" Jul 10 21:04:59.079171 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.079144 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7lzn"] Jul 10 21:04:59.081054 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:04:59.081029 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32440622_a23a_4e8a_9cca_666f88134ee6.slice/crio-60cfc733ac9b342ec8fbceecacc67f5d2b7ee3e05a229737f12a08dcdbf130af WatchSource:0}: Error finding container 60cfc733ac9b342ec8fbceecacc67f5d2b7ee3e05a229737f12a08dcdbf130af: Status 404 returned error can't find the container with id 60cfc733ac9b342ec8fbceecacc67f5d2b7ee3e05a229737f12a08dcdbf130af Jul 10 21:04:59.737785 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.737759 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dzzq4" Jul 10 21:04:59.737785 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.737773 2566 scope.go:117] "RemoveContainer" containerID="d91670d67c2be9de663b9c7b6146e49c19b0d94197f29d9c9631fdbcef80ff9d" Jul 10 21:04:59.739926 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.739693 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7lzn" event={"ID":"32440622-a23a-4e8a-9cca-666f88134ee6","Type":"ContainerStarted","Data":"e03c4cc281583c73180a003234e0546419a1f3570d5b10a01b9ffd57f7888eed"} Jul 10 21:04:59.739926 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.739747 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7lzn" event={"ID":"32440622-a23a-4e8a-9cca-666f88134ee6","Type":"ContainerStarted","Data":"60cfc733ac9b342ec8fbceecacc67f5d2b7ee3e05a229737f12a08dcdbf130af"} Jul 10 21:04:59.739926 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.739805 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7lzn" Jul 10 21:04:59.741601 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.741574 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" event={"ID":"d008e27f-58f3-4294-9f36-1976ba653620","Type":"ContainerStarted","Data":"33f76449b3d8dd858436f63e6c80805bdd501cae67a45f9bde227aaabd852bb4"} Jul 10 21:04:59.741728 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.741619 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" event={"ID":"d008e27f-58f3-4294-9f36-1976ba653620","Type":"ContainerStarted","Data":"c60fa68b087fbaf9bd61c493685ce775a0a22d4654aba42d9f2628dd0e942137"} Jul 10 21:04:59.741728 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.741649 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" Jul 10 21:04:59.744291 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.744263 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" event={"ID":"4cd291af-f4d8-400b-9731-76c483d7f1e7","Type":"ContainerStarted","Data":"10729900b5598b2cfc01930d3b4deb3a49dffdb2000cfbeb54422054750ff446"} Jul 10 21:04:59.744436 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.744294 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:59.744436 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.744309 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:04:59.744436 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.744314 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" event={"ID":"4cd291af-f4d8-400b-9731-76c483d7f1e7","Type":"ContainerStarted","Data":"12daae2b8e7ad481837d0fee48696249a97e87b6c94229d551c74eba5ef7b001"} Jul 10 21:04:59.744436 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.744420 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" Jul 10 21:04:59.764492 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.764439 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7lzn" podStartSLOduration=1.764392279 podStartE2EDuration="1.764392279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:04:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:04:59.762593282 +0000 UTC m=+433.346288885" watchObservedRunningTime="2026-07-10 21:04:59.764392279 +0000 UTC m=+433.348087957" Jul 10 21:04:59.800539 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.800492 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" podStartSLOduration=1.800475391 podStartE2EDuration="1.800475391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:04:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:04:59.799329811 +0000 UTC m=+433.383025413" watchObservedRunningTime="2026-07-10 21:04:59.800475391 +0000 UTC m=+433.384170983" Jul 10 21:04:59.829344 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:04:59.829285 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" podStartSLOduration=1.829270745 podStartE2EDuration="1.829270745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:04:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:04:59.828079383 +0000 UTC m=+433.411774985" watchObservedRunningTime="2026-07-10 21:04:59.829270745 +0000 UTC m=+433.412966346" Jul 10 21:04:59.830724 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:59.830700 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5wktl\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="b027002c-fd25-4009-8831-a752553a516a" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5wktl" Jul 10 21:04:59.832441 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:04:59.832389 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-spbvn\" is forbidden: User \"system:node:ip-10-0-131-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-227.ec2.internal' and this object" podUID="fedea228-efc3-4da0-9a19-fa56b6c84818" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-spbvn" Jul 10 21:05:10.753002 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:10.752913 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" Jul 10 21:05:10.753002 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:10.753000 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-j7lzn" Jul 10 21:05:10.753629 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:10.753158 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" Jul 10 21:05:10.855943 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:10.855909 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m"] Jul 10 21:05:10.856124 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:10.856102 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" podUID="d008e27f-58f3-4294-9f36-1976ba653620" containerName="manager" containerID="cri-o://33f76449b3d8dd858436f63e6c80805bdd501cae67a45f9bde227aaabd852bb4" gracePeriod=10 Jul 10 21:05:11.103338 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.103312 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" Jul 10 21:05:11.204752 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.204719 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxfqn\" (UniqueName: \"kubernetes.io/projected/d008e27f-58f3-4294-9f36-1976ba653620-kube-api-access-qxfqn\") pod \"d008e27f-58f3-4294-9f36-1976ba653620\" (UID: \"d008e27f-58f3-4294-9f36-1976ba653620\") " Jul 10 21:05:11.204919 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.204770 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d008e27f-58f3-4294-9f36-1976ba653620-extensions-socket-volume\") pod \"d008e27f-58f3-4294-9f36-1976ba653620\" (UID: \"d008e27f-58f3-4294-9f36-1976ba653620\") " Jul 10 21:05:11.205189 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.205159 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d008e27f-58f3-4294-9f36-1976ba653620-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d008e27f-58f3-4294-9f36-1976ba653620" (UID: "d008e27f-58f3-4294-9f36-1976ba653620"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:05:11.206886 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.206861 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d008e27f-58f3-4294-9f36-1976ba653620-kube-api-access-qxfqn" (OuterVolumeSpecName: "kube-api-access-qxfqn") pod "d008e27f-58f3-4294-9f36-1976ba653620" (UID: "d008e27f-58f3-4294-9f36-1976ba653620"). InnerVolumeSpecName "kube-api-access-qxfqn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:05:11.305524 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.305444 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxfqn\" (UniqueName: \"kubernetes.io/projected/d008e27f-58f3-4294-9f36-1976ba653620-kube-api-access-qxfqn\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:05:11.305524 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.305473 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d008e27f-58f3-4294-9f36-1976ba653620-extensions-socket-volume\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:05:11.338606 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.338562 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks"] Jul 10 21:05:11.338889 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.338875 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d008e27f-58f3-4294-9f36-1976ba653620" containerName="manager" Jul 10 21:05:11.338889 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.338889 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d008e27f-58f3-4294-9f36-1976ba653620" containerName="manager" Jul 10 21:05:11.338969 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.338946 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d008e27f-58f3-4294-9f36-1976ba653620" containerName="manager" Jul 10 21:05:11.342200 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.342183 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" Jul 10 21:05:11.364935 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.364897 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks"] Jul 10 21:05:11.406734 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.406703 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2k79\" (UniqueName: \"kubernetes.io/projected/2a17d474-5464-4ed5-ac83-f269ef2d50d3-kube-api-access-r2k79\") pod \"kuadrant-operator-controller-manager-86878c769f-ldlks\" (UID: \"2a17d474-5464-4ed5-ac83-f269ef2d50d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" Jul 10 21:05:11.406908 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.406764 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a17d474-5464-4ed5-ac83-f269ef2d50d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ldlks\" (UID: \"2a17d474-5464-4ed5-ac83-f269ef2d50d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" Jul 10 21:05:11.507658 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.507625 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2k79\" (UniqueName: \"kubernetes.io/projected/2a17d474-5464-4ed5-ac83-f269ef2d50d3-kube-api-access-r2k79\") pod \"kuadrant-operator-controller-manager-86878c769f-ldlks\" (UID: \"2a17d474-5464-4ed5-ac83-f269ef2d50d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" Jul 10 21:05:11.507839 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.507681 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a17d474-5464-4ed5-ac83-f269ef2d50d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ldlks\" (UID: \"2a17d474-5464-4ed5-ac83-f269ef2d50d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" Jul 10 21:05:11.508012 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.507996 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a17d474-5464-4ed5-ac83-f269ef2d50d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ldlks\" (UID: \"2a17d474-5464-4ed5-ac83-f269ef2d50d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" Jul 10 21:05:11.519626 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.519576 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2k79\" (UniqueName: \"kubernetes.io/projected/2a17d474-5464-4ed5-ac83-f269ef2d50d3-kube-api-access-r2k79\") pod \"kuadrant-operator-controller-manager-86878c769f-ldlks\" (UID: \"2a17d474-5464-4ed5-ac83-f269ef2d50d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" Jul 10 21:05:11.651710 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.651619 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" Jul 10 21:05:11.783884 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.783859 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks"] Jul 10 21:05:11.786508 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:05:11.786483 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a17d474_5464_4ed5_ac83_f269ef2d50d3.slice/crio-19b12dc950e68a56b240f76ad68c74f4c54ad6c5968ff267fd0e15ab78fdfa85 WatchSource:0}: Error finding container 19b12dc950e68a56b240f76ad68c74f4c54ad6c5968ff267fd0e15ab78fdfa85: Status 404 returned error can't find the container with id 19b12dc950e68a56b240f76ad68c74f4c54ad6c5968ff267fd0e15ab78fdfa85 Jul 10 21:05:11.801562 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.801536 2566 generic.go:356] "Generic (PLEG): container finished" podID="d008e27f-58f3-4294-9f36-1976ba653620" containerID="33f76449b3d8dd858436f63e6c80805bdd501cae67a45f9bde227aaabd852bb4" exitCode=0 Jul 10 21:05:11.801669 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.801574 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" Jul 10 21:05:11.801669 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.801613 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" event={"ID":"d008e27f-58f3-4294-9f36-1976ba653620","Type":"ContainerDied","Data":"33f76449b3d8dd858436f63e6c80805bdd501cae67a45f9bde227aaabd852bb4"} Jul 10 21:05:11.801761 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.801680 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m" event={"ID":"d008e27f-58f3-4294-9f36-1976ba653620","Type":"ContainerDied","Data":"c60fa68b087fbaf9bd61c493685ce775a0a22d4654aba42d9f2628dd0e942137"} Jul 10 21:05:11.801761 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.801708 2566 scope.go:117] "RemoveContainer" containerID="33f76449b3d8dd858436f63e6c80805bdd501cae67a45f9bde227aaabd852bb4" Jul 10 21:05:11.803197 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.803177 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" event={"ID":"2a17d474-5464-4ed5-ac83-f269ef2d50d3","Type":"ContainerStarted","Data":"19b12dc950e68a56b240f76ad68c74f4c54ad6c5968ff267fd0e15ab78fdfa85"} Jul 10 21:05:11.810922 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.810895 2566 scope.go:117] "RemoveContainer" containerID="33f76449b3d8dd858436f63e6c80805bdd501cae67a45f9bde227aaabd852bb4" Jul 10 21:05:11.811206 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:05:11.811187 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33f76449b3d8dd858436f63e6c80805bdd501cae67a45f9bde227aaabd852bb4\": container with ID starting with 33f76449b3d8dd858436f63e6c80805bdd501cae67a45f9bde227aaabd852bb4 not found: ID does not exist" containerID="33f76449b3d8dd858436f63e6c80805bdd501cae67a45f9bde227aaabd852bb4" Jul 10 21:05:11.811267 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.811214 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33f76449b3d8dd858436f63e6c80805bdd501cae67a45f9bde227aaabd852bb4"} err="failed to get container status \"33f76449b3d8dd858436f63e6c80805bdd501cae67a45f9bde227aaabd852bb4\": rpc error: code = NotFound desc = could not find container \"33f76449b3d8dd858436f63e6c80805bdd501cae67a45f9bde227aaabd852bb4\": container with ID starting with 33f76449b3d8dd858436f63e6c80805bdd501cae67a45f9bde227aaabd852bb4 not found: ID does not exist" Jul 10 21:05:11.836927 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.836902 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m"] Jul 10 21:05:11.845786 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:11.845763 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rsr8m"] Jul 10 21:05:12.810297 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:12.810259 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" event={"ID":"2a17d474-5464-4ed5-ac83-f269ef2d50d3","Type":"ContainerStarted","Data":"b9e64e6ebe48e15680db50277180ee3d3f338ee9946019079d8f506f400d4f31"} Jul 10 21:05:12.810759 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:12.810375 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" Jul 10 21:05:12.832088 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:12.832034 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" podStartSLOduration=1.832017781 podStartE2EDuration="1.832017781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:05:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:05:12.831069005 +0000 UTC m=+446.414764607" watchObservedRunningTime="2026-07-10 21:05:12.832017781 +0000 UTC m=+446.415713383" Jul 10 21:05:13.072711 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:13.068410 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d008e27f-58f3-4294-9f36-1976ba653620" path="/var/lib/kubelet/pods/d008e27f-58f3-4294-9f36-1976ba653620/volumes" Jul 10 21:05:23.817556 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:23.817522 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" Jul 10 21:05:23.866067 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:23.866034 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc"] Jul 10 21:05:23.866304 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:23.866280 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" podUID="4cd291af-f4d8-400b-9731-76c483d7f1e7" containerName="manager" containerID="cri-o://10729900b5598b2cfc01930d3b4deb3a49dffdb2000cfbeb54422054750ff446" gracePeriod=10 Jul 10 21:05:24.118929 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.118903 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" Jul 10 21:05:24.218088 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.218059 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8qcz\" (UniqueName: \"kubernetes.io/projected/4cd291af-f4d8-400b-9731-76c483d7f1e7-kube-api-access-v8qcz\") pod \"4cd291af-f4d8-400b-9731-76c483d7f1e7\" (UID: \"4cd291af-f4d8-400b-9731-76c483d7f1e7\") " Jul 10 21:05:24.218297 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.218107 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4cd291af-f4d8-400b-9731-76c483d7f1e7-extensions-socket-volume\") pod \"4cd291af-f4d8-400b-9731-76c483d7f1e7\" (UID: \"4cd291af-f4d8-400b-9731-76c483d7f1e7\") " Jul 10 21:05:24.218635 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.218607 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cd291af-f4d8-400b-9731-76c483d7f1e7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4cd291af-f4d8-400b-9731-76c483d7f1e7" (UID: "4cd291af-f4d8-400b-9731-76c483d7f1e7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:05:24.220346 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.220310 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cd291af-f4d8-400b-9731-76c483d7f1e7-kube-api-access-v8qcz" (OuterVolumeSpecName: "kube-api-access-v8qcz") pod "4cd291af-f4d8-400b-9731-76c483d7f1e7" (UID: "4cd291af-f4d8-400b-9731-76c483d7f1e7"). InnerVolumeSpecName "kube-api-access-v8qcz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:05:24.319674 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.319634 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8qcz\" (UniqueName: \"kubernetes.io/projected/4cd291af-f4d8-400b-9731-76c483d7f1e7-kube-api-access-v8qcz\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:05:24.319674 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.319666 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4cd291af-f4d8-400b-9731-76c483d7f1e7-extensions-socket-volume\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:05:24.868061 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.868028 2566 generic.go:356] "Generic (PLEG): container finished" podID="4cd291af-f4d8-400b-9731-76c483d7f1e7" containerID="10729900b5598b2cfc01930d3b4deb3a49dffdb2000cfbeb54422054750ff446" exitCode=0 Jul 10 21:05:24.868580 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.868076 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" Jul 10 21:05:24.868580 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.868126 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" event={"ID":"4cd291af-f4d8-400b-9731-76c483d7f1e7","Type":"ContainerDied","Data":"10729900b5598b2cfc01930d3b4deb3a49dffdb2000cfbeb54422054750ff446"} Jul 10 21:05:24.868580 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.868179 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc" event={"ID":"4cd291af-f4d8-400b-9731-76c483d7f1e7","Type":"ContainerDied","Data":"12daae2b8e7ad481837d0fee48696249a97e87b6c94229d551c74eba5ef7b001"} Jul 10 21:05:24.868580 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.868202 2566 scope.go:117] "RemoveContainer" containerID="10729900b5598b2cfc01930d3b4deb3a49dffdb2000cfbeb54422054750ff446" Jul 10 21:05:24.876816 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.876798 2566 scope.go:117] "RemoveContainer" containerID="10729900b5598b2cfc01930d3b4deb3a49dffdb2000cfbeb54422054750ff446" Jul 10 21:05:24.877099 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:05:24.877080 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10729900b5598b2cfc01930d3b4deb3a49dffdb2000cfbeb54422054750ff446\": container with ID starting with 10729900b5598b2cfc01930d3b4deb3a49dffdb2000cfbeb54422054750ff446 not found: ID does not exist" containerID="10729900b5598b2cfc01930d3b4deb3a49dffdb2000cfbeb54422054750ff446" Jul 10 21:05:24.877157 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.877109 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10729900b5598b2cfc01930d3b4deb3a49dffdb2000cfbeb54422054750ff446"} err="failed to get container status \"10729900b5598b2cfc01930d3b4deb3a49dffdb2000cfbeb54422054750ff446\": rpc error: code = NotFound desc = could not find container \"10729900b5598b2cfc01930d3b4deb3a49dffdb2000cfbeb54422054750ff446\": container with ID starting with 10729900b5598b2cfc01930d3b4deb3a49dffdb2000cfbeb54422054750ff446 not found: ID does not exist" Jul 10 21:05:24.890269 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.890238 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc"] Jul 10 21:05:24.896300 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:24.896273 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-ln2sc"] Jul 10 21:05:25.072853 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:25.072802 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cd291af-f4d8-400b-9731-76c483d7f1e7" path="/var/lib/kubelet/pods/4cd291af-f4d8-400b-9731-76c483d7f1e7/volumes" Jul 10 21:05:43.132334 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.132301 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-855lr"] Jul 10 21:05:43.132763 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.132674 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cd291af-f4d8-400b-9731-76c483d7f1e7" containerName="manager" Jul 10 21:05:43.132763 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.132691 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cd291af-f4d8-400b-9731-76c483d7f1e7" containerName="manager" Jul 10 21:05:43.132763 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.132740 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cd291af-f4d8-400b-9731-76c483d7f1e7" containerName="manager" Jul 10 21:05:43.135445 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.135428 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" Jul 10 21:05:43.138270 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.138248 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 21:05:43.138405 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.138248 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tz56h\"" Jul 10 21:05:43.145336 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.145296 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-855lr"] Jul 10 21:05:43.234649 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.234620 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-855lr"] Jul 10 21:05:43.273153 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.273106 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rp4d\" (UniqueName: \"kubernetes.io/projected/b26c1b81-ea9d-4fe7-a150-1bb77be68e8f-kube-api-access-5rp4d\") pod \"limitador-limitador-64c94fd64f-855lr\" (UID: \"b26c1b81-ea9d-4fe7-a150-1bb77be68e8f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" Jul 10 21:05:43.273153 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.273146 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b26c1b81-ea9d-4fe7-a150-1bb77be68e8f-config-file\") pod \"limitador-limitador-64c94fd64f-855lr\" (UID: \"b26c1b81-ea9d-4fe7-a150-1bb77be68e8f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" Jul 10 21:05:43.374376 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.374338 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rp4d\" (UniqueName: \"kubernetes.io/projected/b26c1b81-ea9d-4fe7-a150-1bb77be68e8f-kube-api-access-5rp4d\") pod \"limitador-limitador-64c94fd64f-855lr\" (UID: \"b26c1b81-ea9d-4fe7-a150-1bb77be68e8f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" Jul 10 21:05:43.374376 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.374381 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b26c1b81-ea9d-4fe7-a150-1bb77be68e8f-config-file\") pod \"limitador-limitador-64c94fd64f-855lr\" (UID: \"b26c1b81-ea9d-4fe7-a150-1bb77be68e8f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" Jul 10 21:05:43.374994 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.374974 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b26c1b81-ea9d-4fe7-a150-1bb77be68e8f-config-file\") pod \"limitador-limitador-64c94fd64f-855lr\" (UID: \"b26c1b81-ea9d-4fe7-a150-1bb77be68e8f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" Jul 10 21:05:43.382588 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.382534 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rp4d\" (UniqueName: \"kubernetes.io/projected/b26c1b81-ea9d-4fe7-a150-1bb77be68e8f-kube-api-access-5rp4d\") pod \"limitador-limitador-64c94fd64f-855lr\" (UID: \"b26c1b81-ea9d-4fe7-a150-1bb77be68e8f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" Jul 10 21:05:43.447021 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.446981 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" Jul 10 21:05:43.590131 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.590109 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-855lr"] Jul 10 21:05:43.592918 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:05:43.592889 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb26c1b81_ea9d_4fe7_a150_1bb77be68e8f.slice/crio-81d3ca7d669c8fbb5b93a43d9ac01571f2cff370cf1b7c2c233d8329cb1dab05 WatchSource:0}: Error finding container 81d3ca7d669c8fbb5b93a43d9ac01571f2cff370cf1b7c2c233d8329cb1dab05: Status 404 returned error can't find the container with id 81d3ca7d669c8fbb5b93a43d9ac01571f2cff370cf1b7c2c233d8329cb1dab05 Jul 10 21:05:43.951680 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.951644 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" event={"ID":"b26c1b81-ea9d-4fe7-a150-1bb77be68e8f","Type":"ContainerStarted","Data":"81d3ca7d669c8fbb5b93a43d9ac01571f2cff370cf1b7c2c233d8329cb1dab05"} Jul 10 21:05:43.985922 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.985884 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-n98kb"] Jul 10 21:05:43.990434 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.990391 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n98kb" Jul 10 21:05:43.992740 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.992716 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rwlwg\"" Jul 10 21:05:43.999777 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:43.999748 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-n98kb"] Jul 10 21:05:44.080938 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.080896 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfxfn\" (UniqueName: \"kubernetes.io/projected/e624d959-b209-4aac-9ae7-57dff3fd1d0c-kube-api-access-rfxfn\") pod \"authorino-58cd955675-n98kb\" (UID: \"e624d959-b209-4aac-9ae7-57dff3fd1d0c\") " pod="kuadrant-system/authorino-58cd955675-n98kb" Jul 10 21:05:44.181659 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.181620 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxfn\" (UniqueName: \"kubernetes.io/projected/e624d959-b209-4aac-9ae7-57dff3fd1d0c-kube-api-access-rfxfn\") pod \"authorino-58cd955675-n98kb\" (UID: \"e624d959-b209-4aac-9ae7-57dff3fd1d0c\") " pod="kuadrant-system/authorino-58cd955675-n98kb" Jul 10 21:05:44.191097 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.191064 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfxfn\" (UniqueName: \"kubernetes.io/projected/e624d959-b209-4aac-9ae7-57dff3fd1d0c-kube-api-access-rfxfn\") pod \"authorino-58cd955675-n98kb\" (UID: \"e624d959-b209-4aac-9ae7-57dff3fd1d0c\") " pod="kuadrant-system/authorino-58cd955675-n98kb" Jul 10 21:05:44.192860 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.192837 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-jlftn"] Jul 10 21:05:44.196238 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.196219 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jlftn" Jul 10 21:05:44.204031 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.203963 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jlftn"] Jul 10 21:05:44.301188 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.301152 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n98kb" Jul 10 21:05:44.383726 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.383686 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gvfq\" (UniqueName: \"kubernetes.io/projected/38b4e2a3-8de5-4d80-a786-d88e06400e07-kube-api-access-6gvfq\") pod \"authorino-75d556bd77-jlftn\" (UID: \"38b4e2a3-8de5-4d80-a786-d88e06400e07\") " pod="kuadrant-system/authorino-75d556bd77-jlftn" Jul 10 21:05:44.478660 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.478541 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-n98kb"] Jul 10 21:05:44.484201 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.484144 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gvfq\" (UniqueName: \"kubernetes.io/projected/38b4e2a3-8de5-4d80-a786-d88e06400e07-kube-api-access-6gvfq\") pod \"authorino-75d556bd77-jlftn\" (UID: \"38b4e2a3-8de5-4d80-a786-d88e06400e07\") " pod="kuadrant-system/authorino-75d556bd77-jlftn" Jul 10 21:05:44.494947 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.494421 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gvfq\" (UniqueName: \"kubernetes.io/projected/38b4e2a3-8de5-4d80-a786-d88e06400e07-kube-api-access-6gvfq\") pod \"authorino-75d556bd77-jlftn\" (UID: \"38b4e2a3-8de5-4d80-a786-d88e06400e07\") " pod="kuadrant-system/authorino-75d556bd77-jlftn" Jul 10 21:05:44.508032 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.507999 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jlftn" Jul 10 21:05:44.679789 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.679764 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jlftn"] Jul 10 21:05:44.683153 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:05:44.683119 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38b4e2a3_8de5_4d80_a786_d88e06400e07.slice/crio-5e8f00a360e9f7c1f64446688bd40e3eb2f4dad5da7bf54a7b19988c765c3122 WatchSource:0}: Error finding container 5e8f00a360e9f7c1f64446688bd40e3eb2f4dad5da7bf54a7b19988c765c3122: Status 404 returned error can't find the container with id 5e8f00a360e9f7c1f64446688bd40e3eb2f4dad5da7bf54a7b19988c765c3122 Jul 10 21:05:44.957670 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.957577 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jlftn" event={"ID":"38b4e2a3-8de5-4d80-a786-d88e06400e07","Type":"ContainerStarted","Data":"5e8f00a360e9f7c1f64446688bd40e3eb2f4dad5da7bf54a7b19988c765c3122"} Jul 10 21:05:44.958865 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:44.958838 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n98kb" event={"ID":"e624d959-b209-4aac-9ae7-57dff3fd1d0c","Type":"ContainerStarted","Data":"e96eda5b60dc3aa4a864930d04c37edc007d980f5836b731beb10ee7dd4bfb90"} Jul 10 21:05:48.979329 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:48.979290 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" event={"ID":"b26c1b81-ea9d-4fe7-a150-1bb77be68e8f","Type":"ContainerStarted","Data":"41336af14dfe39e055e995f4d4b3c9b416dd0c025726f735517bea7e798981aa"} Jul 10 21:05:48.979794 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:48.979441 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" Jul 10 21:05:48.980772 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:48.980749 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jlftn" event={"ID":"38b4e2a3-8de5-4d80-a786-d88e06400e07","Type":"ContainerStarted","Data":"e8536a29cab4bab0d6da43db0ea6980d0f713d53a47cf86da34134b923da945e"} Jul 10 21:05:48.981983 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:48.981962 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n98kb" event={"ID":"e624d959-b209-4aac-9ae7-57dff3fd1d0c","Type":"ContainerStarted","Data":"d3ba079bf44282dccf6ebc8244569d6acc50f8f7c3697d9787814c540c09a0a6"} Jul 10 21:05:48.999545 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:48.999508 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" podStartSLOduration=1.050986708 podStartE2EDuration="5.99949499s" totalImagesPullingTime="4.948508282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:05:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:05:48.997951388 +0000 UTC m=+482.581646991" watchObservedRunningTime="2026-07-10 21:05:48.99949499 +0000 UTC m=+482.583190593" Jul 10 21:05:49.025243 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:49.025174 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-n98kb" podStartSLOduration=1.9044210449999999 podStartE2EDuration="6.025131083s" totalImagesPullingTime="4.120710038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:05:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:05:49.022269139 +0000 UTC m=+482.605964741" watchObservedRunningTime="2026-07-10 21:05:49.025131083 +0000 UTC m=+482.608826687" Jul 10 21:05:49.040800 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:49.040756 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-jlftn" podStartSLOduration=1.108766815 podStartE2EDuration="5.040742678s" totalImagesPullingTime="3.931975863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:05:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:05:49.039209163 +0000 UTC m=+482.622904780" watchObservedRunningTime="2026-07-10 21:05:49.040742678 +0000 UTC m=+482.624438280" Jul 10 21:05:49.068918 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:49.068889 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-n98kb"] Jul 10 21:05:50.992405 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:50.992349 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-n98kb" podUID="e624d959-b209-4aac-9ae7-57dff3fd1d0c" containerName="authorino" containerID="cri-o://d3ba079bf44282dccf6ebc8244569d6acc50f8f7c3697d9787814c540c09a0a6" gracePeriod=30 Jul 10 21:05:51.232132 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:51.232104 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n98kb" Jul 10 21:05:51.345076 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:51.344995 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfxfn\" (UniqueName: \"kubernetes.io/projected/e624d959-b209-4aac-9ae7-57dff3fd1d0c-kube-api-access-rfxfn\") pod \"e624d959-b209-4aac-9ae7-57dff3fd1d0c\" (UID: \"e624d959-b209-4aac-9ae7-57dff3fd1d0c\") " Jul 10 21:05:51.347017 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:51.346992 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e624d959-b209-4aac-9ae7-57dff3fd1d0c-kube-api-access-rfxfn" (OuterVolumeSpecName: "kube-api-access-rfxfn") pod "e624d959-b209-4aac-9ae7-57dff3fd1d0c" (UID: "e624d959-b209-4aac-9ae7-57dff3fd1d0c"). InnerVolumeSpecName "kube-api-access-rfxfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:05:51.446499 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:51.446463 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfxfn\" (UniqueName: \"kubernetes.io/projected/e624d959-b209-4aac-9ae7-57dff3fd1d0c-kube-api-access-rfxfn\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:05:51.997432 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:51.997377 2566 generic.go:356] "Generic (PLEG): container finished" podID="e624d959-b209-4aac-9ae7-57dff3fd1d0c" containerID="d3ba079bf44282dccf6ebc8244569d6acc50f8f7c3697d9787814c540c09a0a6" exitCode=0 Jul 10 21:05:51.997432 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:51.997431 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-n98kb" Jul 10 21:05:51.997861 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:51.997467 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n98kb" event={"ID":"e624d959-b209-4aac-9ae7-57dff3fd1d0c","Type":"ContainerDied","Data":"d3ba079bf44282dccf6ebc8244569d6acc50f8f7c3697d9787814c540c09a0a6"} Jul 10 21:05:51.997861 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:51.997514 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-n98kb" event={"ID":"e624d959-b209-4aac-9ae7-57dff3fd1d0c","Type":"ContainerDied","Data":"e96eda5b60dc3aa4a864930d04c37edc007d980f5836b731beb10ee7dd4bfb90"} Jul 10 21:05:51.997861 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:51.997534 2566 scope.go:117] "RemoveContainer" containerID="d3ba079bf44282dccf6ebc8244569d6acc50f8f7c3697d9787814c540c09a0a6" Jul 10 21:05:52.005948 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:52.005931 2566 scope.go:117] "RemoveContainer" containerID="d3ba079bf44282dccf6ebc8244569d6acc50f8f7c3697d9787814c540c09a0a6" Jul 10 21:05:52.006205 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:05:52.006188 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3ba079bf44282dccf6ebc8244569d6acc50f8f7c3697d9787814c540c09a0a6\": container with ID starting with d3ba079bf44282dccf6ebc8244569d6acc50f8f7c3697d9787814c540c09a0a6 not found: ID does not exist" containerID="d3ba079bf44282dccf6ebc8244569d6acc50f8f7c3697d9787814c540c09a0a6" Jul 10 21:05:52.006246 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:52.006215 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3ba079bf44282dccf6ebc8244569d6acc50f8f7c3697d9787814c540c09a0a6"} err="failed to get container status \"d3ba079bf44282dccf6ebc8244569d6acc50f8f7c3697d9787814c540c09a0a6\": rpc error: code = NotFound desc = could not find container \"d3ba079bf44282dccf6ebc8244569d6acc50f8f7c3697d9787814c540c09a0a6\": container with ID starting with d3ba079bf44282dccf6ebc8244569d6acc50f8f7c3697d9787814c540c09a0a6 not found: ID does not exist" Jul 10 21:05:52.021207 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:52.021174 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-n98kb"] Jul 10 21:05:52.027568 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:52.027544 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-n98kb"] Jul 10 21:05:53.067797 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:53.067763 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e624d959-b209-4aac-9ae7-57dff3fd1d0c" path="/var/lib/kubelet/pods/e624d959-b209-4aac-9ae7-57dff3fd1d0c/volumes" Jul 10 21:05:58.447306 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:58.447273 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-855lr"] Jul 10 21:05:58.447833 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:58.447494 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" podUID="b26c1b81-ea9d-4fe7-a150-1bb77be68e8f" containerName="limitador" containerID="cri-o://41336af14dfe39e055e995f4d4b3c9b416dd0c025726f735517bea7e798981aa" gracePeriod=30 Jul 10 21:05:58.448391 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:58.448253 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" Jul 10 21:05:59.002938 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.002905 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" Jul 10 21:05:59.027973 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.027895 2566 generic.go:356] "Generic (PLEG): container finished" podID="b26c1b81-ea9d-4fe7-a150-1bb77be68e8f" containerID="41336af14dfe39e055e995f4d4b3c9b416dd0c025726f735517bea7e798981aa" exitCode=0 Jul 10 21:05:59.027973 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.027938 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" Jul 10 21:05:59.028151 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.027977 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" event={"ID":"b26c1b81-ea9d-4fe7-a150-1bb77be68e8f","Type":"ContainerDied","Data":"41336af14dfe39e055e995f4d4b3c9b416dd0c025726f735517bea7e798981aa"} Jul 10 21:05:59.028151 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.028040 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-855lr" event={"ID":"b26c1b81-ea9d-4fe7-a150-1bb77be68e8f","Type":"ContainerDied","Data":"81d3ca7d669c8fbb5b93a43d9ac01571f2cff370cf1b7c2c233d8329cb1dab05"} Jul 10 21:05:59.028151 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.028073 2566 scope.go:117] "RemoveContainer" containerID="41336af14dfe39e055e995f4d4b3c9b416dd0c025726f735517bea7e798981aa" Jul 10 21:05:59.036693 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.036671 2566 scope.go:117] "RemoveContainer" containerID="41336af14dfe39e055e995f4d4b3c9b416dd0c025726f735517bea7e798981aa" Jul 10 21:05:59.036958 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:05:59.036943 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41336af14dfe39e055e995f4d4b3c9b416dd0c025726f735517bea7e798981aa\": container with ID starting with 41336af14dfe39e055e995f4d4b3c9b416dd0c025726f735517bea7e798981aa not found: ID does not exist" containerID="41336af14dfe39e055e995f4d4b3c9b416dd0c025726f735517bea7e798981aa" Jul 10 21:05:59.037012 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.036968 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41336af14dfe39e055e995f4d4b3c9b416dd0c025726f735517bea7e798981aa"} err="failed to get container status \"41336af14dfe39e055e995f4d4b3c9b416dd0c025726f735517bea7e798981aa\": rpc error: code = NotFound desc = could not find container \"41336af14dfe39e055e995f4d4b3c9b416dd0c025726f735517bea7e798981aa\": container with ID starting with 41336af14dfe39e055e995f4d4b3c9b416dd0c025726f735517bea7e798981aa not found: ID does not exist" Jul 10 21:05:59.110890 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.110866 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rp4d\" (UniqueName: \"kubernetes.io/projected/b26c1b81-ea9d-4fe7-a150-1bb77be68e8f-kube-api-access-5rp4d\") pod \"b26c1b81-ea9d-4fe7-a150-1bb77be68e8f\" (UID: \"b26c1b81-ea9d-4fe7-a150-1bb77be68e8f\") " Jul 10 21:05:59.111045 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.110912 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b26c1b81-ea9d-4fe7-a150-1bb77be68e8f-config-file\") pod \"b26c1b81-ea9d-4fe7-a150-1bb77be68e8f\" (UID: \"b26c1b81-ea9d-4fe7-a150-1bb77be68e8f\") " Jul 10 21:05:59.111279 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.111260 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b26c1b81-ea9d-4fe7-a150-1bb77be68e8f-config-file" (OuterVolumeSpecName: "config-file") pod "b26c1b81-ea9d-4fe7-a150-1bb77be68e8f" (UID: "b26c1b81-ea9d-4fe7-a150-1bb77be68e8f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:05:59.112842 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.112819 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b26c1b81-ea9d-4fe7-a150-1bb77be68e8f-kube-api-access-5rp4d" (OuterVolumeSpecName: "kube-api-access-5rp4d") pod "b26c1b81-ea9d-4fe7-a150-1bb77be68e8f" (UID: "b26c1b81-ea9d-4fe7-a150-1bb77be68e8f"). InnerVolumeSpecName "kube-api-access-5rp4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:05:59.212384 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.212352 2566 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b26c1b81-ea9d-4fe7-a150-1bb77be68e8f-config-file\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:05:59.212384 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.212378 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rp4d\" (UniqueName: \"kubernetes.io/projected/b26c1b81-ea9d-4fe7-a150-1bb77be68e8f-kube-api-access-5rp4d\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:05:59.349589 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.349562 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-855lr"] Jul 10 21:05:59.352030 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:05:59.352010 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-855lr"] Jul 10 21:06:01.067496 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.067465 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b26c1b81-ea9d-4fe7-a150-1bb77be68e8f" path="/var/lib/kubelet/pods/b26c1b81-ea9d-4fe7-a150-1bb77be68e8f/volumes" Jul 10 21:06:01.111243 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.111218 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j"] Jul 10 21:06:01.111651 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.111635 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b26c1b81-ea9d-4fe7-a150-1bb77be68e8f" containerName="limitador" Jul 10 21:06:01.111744 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.111653 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b26c1b81-ea9d-4fe7-a150-1bb77be68e8f" containerName="limitador" Jul 10 21:06:01.111744 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.111666 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e624d959-b209-4aac-9ae7-57dff3fd1d0c" containerName="authorino" Jul 10 21:06:01.111744 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.111674 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e624d959-b209-4aac-9ae7-57dff3fd1d0c" containerName="authorino" Jul 10 21:06:01.111744 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.111741 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e624d959-b209-4aac-9ae7-57dff3fd1d0c" containerName="authorino" Jul 10 21:06:01.111932 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.111755 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b26c1b81-ea9d-4fe7-a150-1bb77be68e8f" containerName="limitador" Jul 10 21:06:01.117337 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.117307 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j" Jul 10 21:06:01.122125 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.122102 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 21:06:01.122691 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.122672 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-44rng\"" Jul 10 21:06:01.122940 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.122918 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 10 21:06:01.123116 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.123094 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 10 21:06:01.128850 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.128825 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j"] Jul 10 21:06:01.230327 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.230292 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6c99a523-0aaa-4d8f-a829-e679b2f1941e-data\") pod \"postgres-69fb4594cf-6hx8j\" (UID: \"6c99a523-0aaa-4d8f-a829-e679b2f1941e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j" Jul 10 21:06:01.230527 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.230345 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mn86\" (UniqueName: \"kubernetes.io/projected/6c99a523-0aaa-4d8f-a829-e679b2f1941e-kube-api-access-6mn86\") pod \"postgres-69fb4594cf-6hx8j\" (UID: \"6c99a523-0aaa-4d8f-a829-e679b2f1941e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j" Jul 10 21:06:01.331355 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.331267 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6c99a523-0aaa-4d8f-a829-e679b2f1941e-data\") pod \"postgres-69fb4594cf-6hx8j\" (UID: \"6c99a523-0aaa-4d8f-a829-e679b2f1941e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j" Jul 10 21:06:01.331355 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.331332 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mn86\" (UniqueName: \"kubernetes.io/projected/6c99a523-0aaa-4d8f-a829-e679b2f1941e-kube-api-access-6mn86\") pod \"postgres-69fb4594cf-6hx8j\" (UID: \"6c99a523-0aaa-4d8f-a829-e679b2f1941e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j" Jul 10 21:06:01.331767 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.331748 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6c99a523-0aaa-4d8f-a829-e679b2f1941e-data\") pod \"postgres-69fb4594cf-6hx8j\" (UID: \"6c99a523-0aaa-4d8f-a829-e679b2f1941e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j" Jul 10 21:06:01.346504 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.346473 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mn86\" (UniqueName: \"kubernetes.io/projected/6c99a523-0aaa-4d8f-a829-e679b2f1941e-kube-api-access-6mn86\") pod \"postgres-69fb4594cf-6hx8j\" (UID: \"6c99a523-0aaa-4d8f-a829-e679b2f1941e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j" Jul 10 21:06:01.431606 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.431571 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j" Jul 10 21:06:01.559140 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:01.559118 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j"] Jul 10 21:06:01.561255 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:06:01.561224 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c99a523_0aaa_4d8f_a829_e679b2f1941e.slice/crio-5eb114fb62f309151f62947f76e49369a2d3d08b361f3a08a07a3702331c36fe WatchSource:0}: Error finding container 5eb114fb62f309151f62947f76e49369a2d3d08b361f3a08a07a3702331c36fe: Status 404 returned error can't find the container with id 5eb114fb62f309151f62947f76e49369a2d3d08b361f3a08a07a3702331c36fe Jul 10 21:06:02.042346 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:02.042313 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j" event={"ID":"6c99a523-0aaa-4d8f-a829-e679b2f1941e","Type":"ContainerStarted","Data":"5eb114fb62f309151f62947f76e49369a2d3d08b361f3a08a07a3702331c36fe"} Jul 10 21:06:07.070711 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:07.070678 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j" event={"ID":"6c99a523-0aaa-4d8f-a829-e679b2f1941e","Type":"ContainerStarted","Data":"a472f98ae7985efaafc5fda3b3d87484c3cdbb0d5a29cdc0c20187f89985315c"} Jul 10 21:06:07.071105 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:07.070810 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j" Jul 10 21:06:07.087895 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:07.087839 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j" podStartSLOduration=1.253045741 podStartE2EDuration="6.08782091s" totalImagesPullingTime="4.834775169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:06:07.08698101 +0000 UTC m=+500.670676626" watchObservedRunningTime="2026-07-10 21:06:07.08782091 +0000 UTC m=+500.671516511" Jul 10 21:06:13.105670 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:13.105640 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6hx8j" Jul 10 21:06:13.639338 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:13.639305 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cj5n4"] Jul 10 21:06:13.642763 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:13.642741 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cj5n4" Jul 10 21:06:13.647386 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:13.647356 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cj5n4"] Jul 10 21:06:13.737702 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:13.737670 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5gvz\" (UniqueName: \"kubernetes.io/projected/824d2470-b566-450a-9ba9-55757553eb3a-kube-api-access-g5gvz\") pod \"authorino-5d4b67c446-cj5n4\" (UID: \"824d2470-b566-450a-9ba9-55757553eb3a\") " pod="kuadrant-system/authorino-5d4b67c446-cj5n4" Jul 10 21:06:13.832548 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:13.832511 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cj5n4"] Jul 10 21:06:13.832834 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:06:13.832808 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-g5gvz], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-cj5n4" podUID="824d2470-b566-450a-9ba9-55757553eb3a" Jul 10 21:06:13.839118 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:13.839085 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5gvz\" (UniqueName: \"kubernetes.io/projected/824d2470-b566-450a-9ba9-55757553eb3a-kube-api-access-g5gvz\") pod \"authorino-5d4b67c446-cj5n4\" (UID: \"824d2470-b566-450a-9ba9-55757553eb3a\") " pod="kuadrant-system/authorino-5d4b67c446-cj5n4" Jul 10 21:06:13.847736 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:13.847709 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5gvz\" (UniqueName: \"kubernetes.io/projected/824d2470-b566-450a-9ba9-55757553eb3a-kube-api-access-g5gvz\") pod \"authorino-5d4b67c446-cj5n4\" (UID: \"824d2470-b566-450a-9ba9-55757553eb3a\") " pod="kuadrant-system/authorino-5d4b67c446-cj5n4" Jul 10 21:06:13.932660 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:13.932582 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65c8788bc6-9hnrs"] Jul 10 21:06:13.938759 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:13.938735 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65c8788bc6-9hnrs" Jul 10 21:06:13.941265 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:13.941242 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 10 21:06:13.944017 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:13.943990 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65c8788bc6-9hnrs"] Jul 10 21:06:14.040464 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:14.040427 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b98e476d-c456-4eac-9dd5-48c2079aabcc-tls-cert\") pod \"authorino-65c8788bc6-9hnrs\" (UID: \"b98e476d-c456-4eac-9dd5-48c2079aabcc\") " pod="kuadrant-system/authorino-65c8788bc6-9hnrs" Jul 10 21:06:14.040672 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:14.040491 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdtz6\" (UniqueName: \"kubernetes.io/projected/b98e476d-c456-4eac-9dd5-48c2079aabcc-kube-api-access-pdtz6\") pod \"authorino-65c8788bc6-9hnrs\" (UID: \"b98e476d-c456-4eac-9dd5-48c2079aabcc\") " pod="kuadrant-system/authorino-65c8788bc6-9hnrs" Jul 10 21:06:14.100458 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:14.100430 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cj5n4" Jul 10 21:06:14.110117 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:14.110089 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cj5n4" Jul 10 21:06:14.141817 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:14.141774 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b98e476d-c456-4eac-9dd5-48c2079aabcc-tls-cert\") pod \"authorino-65c8788bc6-9hnrs\" (UID: \"b98e476d-c456-4eac-9dd5-48c2079aabcc\") " pod="kuadrant-system/authorino-65c8788bc6-9hnrs" Jul 10 21:06:14.142001 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:14.141867 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdtz6\" (UniqueName: \"kubernetes.io/projected/b98e476d-c456-4eac-9dd5-48c2079aabcc-kube-api-access-pdtz6\") pod \"authorino-65c8788bc6-9hnrs\" (UID: \"b98e476d-c456-4eac-9dd5-48c2079aabcc\") " pod="kuadrant-system/authorino-65c8788bc6-9hnrs" Jul 10 21:06:14.144277 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:14.144256 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b98e476d-c456-4eac-9dd5-48c2079aabcc-tls-cert\") pod \"authorino-65c8788bc6-9hnrs\" (UID: \"b98e476d-c456-4eac-9dd5-48c2079aabcc\") " pod="kuadrant-system/authorino-65c8788bc6-9hnrs" Jul 10 21:06:14.149816 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:14.149794 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdtz6\" (UniqueName: \"kubernetes.io/projected/b98e476d-c456-4eac-9dd5-48c2079aabcc-kube-api-access-pdtz6\") pod \"authorino-65c8788bc6-9hnrs\" (UID: \"b98e476d-c456-4eac-9dd5-48c2079aabcc\") " pod="kuadrant-system/authorino-65c8788bc6-9hnrs" Jul 10 21:06:14.242663 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:14.242631 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5gvz\" (UniqueName: \"kubernetes.io/projected/824d2470-b566-450a-9ba9-55757553eb3a-kube-api-access-g5gvz\") pod \"824d2470-b566-450a-9ba9-55757553eb3a\" (UID: \"824d2470-b566-450a-9ba9-55757553eb3a\") " Jul 10 21:06:14.244820 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:14.244791 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/824d2470-b566-450a-9ba9-55757553eb3a-kube-api-access-g5gvz" (OuterVolumeSpecName: "kube-api-access-g5gvz") pod "824d2470-b566-450a-9ba9-55757553eb3a" (UID: "824d2470-b566-450a-9ba9-55757553eb3a"). InnerVolumeSpecName "kube-api-access-g5gvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:06:14.254984 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:14.254961 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65c8788bc6-9hnrs" Jul 10 21:06:14.343931 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:14.343902 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5gvz\" (UniqueName: \"kubernetes.io/projected/824d2470-b566-450a-9ba9-55757553eb3a-kube-api-access-g5gvz\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:06:14.410002 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:14.409947 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65c8788bc6-9hnrs"] Jul 10 21:06:14.412935 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:06:14.412903 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb98e476d_c456_4eac_9dd5_48c2079aabcc.slice/crio-7e2cfb5288b2a72dd1d2a00d09a2303954417b3a0fc0663487ff782abda10854 WatchSource:0}: Error finding container 7e2cfb5288b2a72dd1d2a00d09a2303954417b3a0fc0663487ff782abda10854: Status 404 returned error can't find the container with id 7e2cfb5288b2a72dd1d2a00d09a2303954417b3a0fc0663487ff782abda10854 Jul 10 21:06:15.108102 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:15.108079 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cj5n4" Jul 10 21:06:15.108281 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:15.108075 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65c8788bc6-9hnrs" event={"ID":"b98e476d-c456-4eac-9dd5-48c2079aabcc","Type":"ContainerStarted","Data":"7e2cfb5288b2a72dd1d2a00d09a2303954417b3a0fc0663487ff782abda10854"} Jul 10 21:06:15.135743 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:15.135713 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cj5n4"] Jul 10 21:06:15.138998 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:15.138968 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cj5n4"] Jul 10 21:06:16.114522 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:16.114487 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65c8788bc6-9hnrs" event={"ID":"b98e476d-c456-4eac-9dd5-48c2079aabcc","Type":"ContainerStarted","Data":"391fae72117a7f1b1a81fb26faf2994eea4a51b1222ec61dbf04343be1d33d51"} Jul 10 21:06:16.138241 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:16.138189 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-65c8788bc6-9hnrs" podStartSLOduration=2.545172887 podStartE2EDuration="3.138170957s" totalImagesPullingTime="592.99807ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:06:16.13681076 +0000 UTC m=+509.720506356" watchObservedRunningTime="2026-07-10 21:06:16.138170957 +0000 UTC m=+509.721866558" Jul 10 21:06:16.179334 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:16.179294 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jlftn"] Jul 10 21:06:16.179536 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:16.179513 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-jlftn" podUID="38b4e2a3-8de5-4d80-a786-d88e06400e07" containerName="authorino" containerID="cri-o://e8536a29cab4bab0d6da43db0ea6980d0f713d53a47cf86da34134b923da945e" gracePeriod=30 Jul 10 21:06:16.498977 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:16.498945 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jlftn" Jul 10 21:06:16.563385 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:16.563356 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gvfq\" (UniqueName: \"kubernetes.io/projected/38b4e2a3-8de5-4d80-a786-d88e06400e07-kube-api-access-6gvfq\") pod \"38b4e2a3-8de5-4d80-a786-d88e06400e07\" (UID: \"38b4e2a3-8de5-4d80-a786-d88e06400e07\") " Jul 10 21:06:16.565548 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:16.565512 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38b4e2a3-8de5-4d80-a786-d88e06400e07-kube-api-access-6gvfq" (OuterVolumeSpecName: "kube-api-access-6gvfq") pod "38b4e2a3-8de5-4d80-a786-d88e06400e07" (UID: "38b4e2a3-8de5-4d80-a786-d88e06400e07"). InnerVolumeSpecName "kube-api-access-6gvfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:06:16.664092 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:16.664010 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gvfq\" (UniqueName: \"kubernetes.io/projected/38b4e2a3-8de5-4d80-a786-d88e06400e07-kube-api-access-6gvfq\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:06:17.068108 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:17.068074 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="824d2470-b566-450a-9ba9-55757553eb3a" path="/var/lib/kubelet/pods/824d2470-b566-450a-9ba9-55757553eb3a/volumes" Jul 10 21:06:17.119647 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:17.119607 2566 generic.go:356] "Generic (PLEG): container finished" podID="38b4e2a3-8de5-4d80-a786-d88e06400e07" containerID="e8536a29cab4bab0d6da43db0ea6980d0f713d53a47cf86da34134b923da945e" exitCode=0 Jul 10 21:06:17.119647 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:17.119652 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jlftn" Jul 10 21:06:17.119820 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:17.119692 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jlftn" event={"ID":"38b4e2a3-8de5-4d80-a786-d88e06400e07","Type":"ContainerDied","Data":"e8536a29cab4bab0d6da43db0ea6980d0f713d53a47cf86da34134b923da945e"} Jul 10 21:06:17.119820 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:17.119735 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jlftn" event={"ID":"38b4e2a3-8de5-4d80-a786-d88e06400e07","Type":"ContainerDied","Data":"5e8f00a360e9f7c1f64446688bd40e3eb2f4dad5da7bf54a7b19988c765c3122"} Jul 10 21:06:17.119820 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:17.119752 2566 scope.go:117] "RemoveContainer" containerID="e8536a29cab4bab0d6da43db0ea6980d0f713d53a47cf86da34134b923da945e" Jul 10 21:06:17.128343 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:17.128323 2566 scope.go:117] "RemoveContainer" containerID="e8536a29cab4bab0d6da43db0ea6980d0f713d53a47cf86da34134b923da945e" Jul 10 21:06:17.128689 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:06:17.128670 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8536a29cab4bab0d6da43db0ea6980d0f713d53a47cf86da34134b923da945e\": container with ID starting with e8536a29cab4bab0d6da43db0ea6980d0f713d53a47cf86da34134b923da945e not found: ID does not exist" containerID="e8536a29cab4bab0d6da43db0ea6980d0f713d53a47cf86da34134b923da945e" Jul 10 21:06:17.128754 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:17.128701 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8536a29cab4bab0d6da43db0ea6980d0f713d53a47cf86da34134b923da945e"} err="failed to get container status \"e8536a29cab4bab0d6da43db0ea6980d0f713d53a47cf86da34134b923da945e\": rpc error: code = NotFound desc = could not find container \"e8536a29cab4bab0d6da43db0ea6980d0f713d53a47cf86da34134b923da945e\": container with ID starting with e8536a29cab4bab0d6da43db0ea6980d0f713d53a47cf86da34134b923da945e not found: ID does not exist" Jul 10 21:06:17.138820 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:17.138787 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jlftn"] Jul 10 21:06:17.140745 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:17.140720 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jlftn"] Jul 10 21:06:19.067869 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:19.067833 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38b4e2a3-8de5-4d80-a786-d88e06400e07" path="/var/lib/kubelet/pods/38b4e2a3-8de5-4d80-a786-d88e06400e07/volumes" Jul 10 21:06:20.797102 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:20.797071 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f586fdfd-8c4jv"] Jul 10 21:06:20.797596 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:20.797378 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38b4e2a3-8de5-4d80-a786-d88e06400e07" containerName="authorino" Jul 10 21:06:20.797596 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:20.797388 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="38b4e2a3-8de5-4d80-a786-d88e06400e07" containerName="authorino" Jul 10 21:06:20.797596 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:20.797502 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="38b4e2a3-8de5-4d80-a786-d88e06400e07" containerName="authorino" Jul 10 21:06:20.804677 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:20.804653 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f586fdfd-8c4jv" Jul 10 21:06:20.811333 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:20.810588 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 10 21:06:20.811333 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:20.810923 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 21:06:20.811333 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:20.811198 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sfcct\"" Jul 10 21:06:20.820032 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:20.820006 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f586fdfd-8c4jv"] Jul 10 21:06:20.900567 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:20.900533 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c745ebd4-ca73-4d5f-a728-57d228e10177-webhook-cert\") pod \"maas-controller-7f586fdfd-8c4jv\" (UID: \"c745ebd4-ca73-4d5f-a728-57d228e10177\") " pod="opendatahub/maas-controller-7f586fdfd-8c4jv" Jul 10 21:06:20.900745 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:20.900601 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv448\" (UniqueName: \"kubernetes.io/projected/c745ebd4-ca73-4d5f-a728-57d228e10177-kube-api-access-nv448\") pod \"maas-controller-7f586fdfd-8c4jv\" (UID: \"c745ebd4-ca73-4d5f-a728-57d228e10177\") " pod="opendatahub/maas-controller-7f586fdfd-8c4jv" Jul 10 21:06:20.982478 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:20.982445 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5db9df9f4b-25w8g"] Jul 10 21:06:20.984588 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:20.984572 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5db9df9f4b-25w8g" Jul 10 21:06:20.998326 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:20.998297 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5db9df9f4b-25w8g"] Jul 10 21:06:21.001934 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.001912 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c745ebd4-ca73-4d5f-a728-57d228e10177-webhook-cert\") pod \"maas-controller-7f586fdfd-8c4jv\" (UID: \"c745ebd4-ca73-4d5f-a728-57d228e10177\") " pod="opendatahub/maas-controller-7f586fdfd-8c4jv" Jul 10 21:06:21.002051 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.001970 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv448\" (UniqueName: \"kubernetes.io/projected/c745ebd4-ca73-4d5f-a728-57d228e10177-kube-api-access-nv448\") pod \"maas-controller-7f586fdfd-8c4jv\" (UID: \"c745ebd4-ca73-4d5f-a728-57d228e10177\") " pod="opendatahub/maas-controller-7f586fdfd-8c4jv" Jul 10 21:06:21.002113 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:06:21.002074 2566 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 10 21:06:21.002167 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:06:21.002157 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c745ebd4-ca73-4d5f-a728-57d228e10177-webhook-cert podName:c745ebd4-ca73-4d5f-a728-57d228e10177 nodeName:}" failed. No retries permitted until 2026-07-10 21:06:21.502135001 +0000 UTC m=+515.085830582 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/c745ebd4-ca73-4d5f-a728-57d228e10177-webhook-cert") pod "maas-controller-7f586fdfd-8c4jv" (UID: "c745ebd4-ca73-4d5f-a728-57d228e10177") : secret "maas-controller-webhook-cert" not found Jul 10 21:06:21.011499 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.011472 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv448\" (UniqueName: \"kubernetes.io/projected/c745ebd4-ca73-4d5f-a728-57d228e10177-kube-api-access-nv448\") pod \"maas-controller-7f586fdfd-8c4jv\" (UID: \"c745ebd4-ca73-4d5f-a728-57d228e10177\") " pod="opendatahub/maas-controller-7f586fdfd-8c4jv" Jul 10 21:06:21.102850 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.102750 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7cc230d-ec39-4361-9cec-a5866709e755-webhook-cert\") pod \"maas-controller-5db9df9f4b-25w8g\" (UID: \"c7cc230d-ec39-4361-9cec-a5866709e755\") " pod="opendatahub/maas-controller-5db9df9f4b-25w8g" Jul 10 21:06:21.102850 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.102831 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv86j\" (UniqueName: \"kubernetes.io/projected/c7cc230d-ec39-4361-9cec-a5866709e755-kube-api-access-nv86j\") pod \"maas-controller-5db9df9f4b-25w8g\" (UID: \"c7cc230d-ec39-4361-9cec-a5866709e755\") " pod="opendatahub/maas-controller-5db9df9f4b-25w8g" Jul 10 21:06:21.204259 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.204216 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv86j\" (UniqueName: \"kubernetes.io/projected/c7cc230d-ec39-4361-9cec-a5866709e755-kube-api-access-nv86j\") pod \"maas-controller-5db9df9f4b-25w8g\" (UID: \"c7cc230d-ec39-4361-9cec-a5866709e755\") " pod="opendatahub/maas-controller-5db9df9f4b-25w8g" Jul 10 21:06:21.204483 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.204312 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7cc230d-ec39-4361-9cec-a5866709e755-webhook-cert\") pod \"maas-controller-5db9df9f4b-25w8g\" (UID: \"c7cc230d-ec39-4361-9cec-a5866709e755\") " pod="opendatahub/maas-controller-5db9df9f4b-25w8g" Jul 10 21:06:21.206764 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.206744 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7cc230d-ec39-4361-9cec-a5866709e755-webhook-cert\") pod \"maas-controller-5db9df9f4b-25w8g\" (UID: \"c7cc230d-ec39-4361-9cec-a5866709e755\") " pod="opendatahub/maas-controller-5db9df9f4b-25w8g" Jul 10 21:06:21.213456 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.213429 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv86j\" (UniqueName: \"kubernetes.io/projected/c7cc230d-ec39-4361-9cec-a5866709e755-kube-api-access-nv86j\") pod \"maas-controller-5db9df9f4b-25w8g\" (UID: \"c7cc230d-ec39-4361-9cec-a5866709e755\") " pod="opendatahub/maas-controller-5db9df9f4b-25w8g" Jul 10 21:06:21.294504 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.294462 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5db9df9f4b-25w8g" Jul 10 21:06:21.432347 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.432318 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5db9df9f4b-25w8g"] Jul 10 21:06:21.434273 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:06:21.434246 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7cc230d_ec39_4361_9cec_a5866709e755.slice/crio-f1e42f861eb9462cfba027e060a2a739f89cbc32e0bf81ac84cc936a89ce2991 WatchSource:0}: Error finding container f1e42f861eb9462cfba027e060a2a739f89cbc32e0bf81ac84cc936a89ce2991: Status 404 returned error can't find the container with id f1e42f861eb9462cfba027e060a2a739f89cbc32e0bf81ac84cc936a89ce2991 Jul 10 21:06:21.507590 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.507549 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c745ebd4-ca73-4d5f-a728-57d228e10177-webhook-cert\") pod \"maas-controller-7f586fdfd-8c4jv\" (UID: \"c745ebd4-ca73-4d5f-a728-57d228e10177\") " pod="opendatahub/maas-controller-7f586fdfd-8c4jv" Jul 10 21:06:21.509953 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.509934 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c745ebd4-ca73-4d5f-a728-57d228e10177-webhook-cert\") pod \"maas-controller-7f586fdfd-8c4jv\" (UID: \"c745ebd4-ca73-4d5f-a728-57d228e10177\") " pod="opendatahub/maas-controller-7f586fdfd-8c4jv" Jul 10 21:06:21.714974 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.714945 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f586fdfd-8c4jv" Jul 10 21:06:21.850587 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:21.850560 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f586fdfd-8c4jv"] Jul 10 21:06:21.852280 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:06:21.852253 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc745ebd4_ca73_4d5f_a728_57d228e10177.slice/crio-705f44f83e19e810a1bf21b5e5841cdda704366e33a6a06b4c1b7fddb7fe5776 WatchSource:0}: Error finding container 705f44f83e19e810a1bf21b5e5841cdda704366e33a6a06b4c1b7fddb7fe5776: Status 404 returned error can't find the container with id 705f44f83e19e810a1bf21b5e5841cdda704366e33a6a06b4c1b7fddb7fe5776 Jul 10 21:06:22.145359 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:22.145231 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f586fdfd-8c4jv" event={"ID":"c745ebd4-ca73-4d5f-a728-57d228e10177","Type":"ContainerStarted","Data":"705f44f83e19e810a1bf21b5e5841cdda704366e33a6a06b4c1b7fddb7fe5776"} Jul 10 21:06:22.146775 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:22.146741 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5db9df9f4b-25w8g" event={"ID":"c7cc230d-ec39-4361-9cec-a5866709e755","Type":"ContainerStarted","Data":"f1e42f861eb9462cfba027e060a2a739f89cbc32e0bf81ac84cc936a89ce2991"} Jul 10 21:06:24.159388 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:24.159350 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5db9df9f4b-25w8g" event={"ID":"c7cc230d-ec39-4361-9cec-a5866709e755","Type":"ContainerStarted","Data":"2bcf0a057a91fb4adfe24dee8a959f95e253cd09030d4ea65fecadc65b90108a"} Jul 10 21:06:24.159915 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:24.159465 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5db9df9f4b-25w8g" Jul 10 21:06:24.160990 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:24.160964 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f586fdfd-8c4jv" event={"ID":"c745ebd4-ca73-4d5f-a728-57d228e10177","Type":"ContainerStarted","Data":"4269760823e842fbd01364dfff460d2450d6a439b9e4f086a1207c3a77ba5729"} Jul 10 21:06:24.161118 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:24.161103 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f586fdfd-8c4jv" Jul 10 21:06:24.182832 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:24.182774 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5db9df9f4b-25w8g" podStartSLOduration=2.070176436 podStartE2EDuration="4.182756713s" totalImagesPullingTime="2.112580277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:06:24.181440281 +0000 UTC m=+517.765135874" watchObservedRunningTime="2026-07-10 21:06:24.182756713 +0000 UTC m=+517.766452315" Jul 10 21:06:24.200284 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:24.200228 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f586fdfd-8c4jv" podStartSLOduration=2.505655771 podStartE2EDuration="4.200210721s" totalImagesPullingTime="1.69455495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:06:24.199281701 +0000 UTC m=+517.782977332" watchObservedRunningTime="2026-07-10 21:06:24.200210721 +0000 UTC m=+517.783906322" Jul 10 21:06:35.171345 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:35.171310 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f586fdfd-8c4jv" Jul 10 21:06:35.171858 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:35.171391 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5db9df9f4b-25w8g" Jul 10 21:06:35.241230 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:35.241191 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f586fdfd-8c4jv"] Jul 10 21:06:35.241464 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:35.241421 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7f586fdfd-8c4jv" podUID="c745ebd4-ca73-4d5f-a728-57d228e10177" containerName="manager" containerID="cri-o://4269760823e842fbd01364dfff460d2450d6a439b9e4f086a1207c3a77ba5729" gracePeriod=30 Jul 10 21:06:35.487746 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:35.487721 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f586fdfd-8c4jv" Jul 10 21:06:35.634899 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:35.634863 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv448\" (UniqueName: \"kubernetes.io/projected/c745ebd4-ca73-4d5f-a728-57d228e10177-kube-api-access-nv448\") pod \"c745ebd4-ca73-4d5f-a728-57d228e10177\" (UID: \"c745ebd4-ca73-4d5f-a728-57d228e10177\") " Jul 10 21:06:35.635107 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:35.634944 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c745ebd4-ca73-4d5f-a728-57d228e10177-webhook-cert\") pod \"c745ebd4-ca73-4d5f-a728-57d228e10177\" (UID: \"c745ebd4-ca73-4d5f-a728-57d228e10177\") " Jul 10 21:06:35.637152 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:35.637120 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c745ebd4-ca73-4d5f-a728-57d228e10177-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c745ebd4-ca73-4d5f-a728-57d228e10177" (UID: "c745ebd4-ca73-4d5f-a728-57d228e10177"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 21:06:35.637260 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:35.637171 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c745ebd4-ca73-4d5f-a728-57d228e10177-kube-api-access-nv448" (OuterVolumeSpecName: "kube-api-access-nv448") pod "c745ebd4-ca73-4d5f-a728-57d228e10177" (UID: "c745ebd4-ca73-4d5f-a728-57d228e10177"). InnerVolumeSpecName "kube-api-access-nv448". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:06:35.735768 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:35.735686 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv448\" (UniqueName: \"kubernetes.io/projected/c745ebd4-ca73-4d5f-a728-57d228e10177-kube-api-access-nv448\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:06:35.735768 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:35.735713 2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c745ebd4-ca73-4d5f-a728-57d228e10177-webhook-cert\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:06:36.216753 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:36.216721 2566 generic.go:356] "Generic (PLEG): container finished" podID="c745ebd4-ca73-4d5f-a728-57d228e10177" containerID="4269760823e842fbd01364dfff460d2450d6a439b9e4f086a1207c3a77ba5729" exitCode=0 Jul 10 21:06:36.217147 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:36.216782 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f586fdfd-8c4jv" Jul 10 21:06:36.217147 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:36.216800 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f586fdfd-8c4jv" event={"ID":"c745ebd4-ca73-4d5f-a728-57d228e10177","Type":"ContainerDied","Data":"4269760823e842fbd01364dfff460d2450d6a439b9e4f086a1207c3a77ba5729"} Jul 10 21:06:36.217147 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:36.216865 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f586fdfd-8c4jv" event={"ID":"c745ebd4-ca73-4d5f-a728-57d228e10177","Type":"ContainerDied","Data":"705f44f83e19e810a1bf21b5e5841cdda704366e33a6a06b4c1b7fddb7fe5776"} Jul 10 21:06:36.217147 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:36.216887 2566 scope.go:117] "RemoveContainer" containerID="4269760823e842fbd01364dfff460d2450d6a439b9e4f086a1207c3a77ba5729" Jul 10 21:06:36.226086 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:36.226068 2566 scope.go:117] "RemoveContainer" containerID="4269760823e842fbd01364dfff460d2450d6a439b9e4f086a1207c3a77ba5729" Jul 10 21:06:36.226335 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:06:36.226316 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4269760823e842fbd01364dfff460d2450d6a439b9e4f086a1207c3a77ba5729\": container with ID starting with 4269760823e842fbd01364dfff460d2450d6a439b9e4f086a1207c3a77ba5729 not found: ID does not exist" containerID="4269760823e842fbd01364dfff460d2450d6a439b9e4f086a1207c3a77ba5729" Jul 10 21:06:36.226383 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:36.226343 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4269760823e842fbd01364dfff460d2450d6a439b9e4f086a1207c3a77ba5729"} err="failed to get container status \"4269760823e842fbd01364dfff460d2450d6a439b9e4f086a1207c3a77ba5729\": rpc error: code = NotFound desc = could not find container \"4269760823e842fbd01364dfff460d2450d6a439b9e4f086a1207c3a77ba5729\": container with ID starting with 4269760823e842fbd01364dfff460d2450d6a439b9e4f086a1207c3a77ba5729 not found: ID does not exist" Jul 10 21:06:36.239818 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:36.239794 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f586fdfd-8c4jv"] Jul 10 21:06:36.243044 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:36.243018 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f586fdfd-8c4jv"] Jul 10 21:06:37.069047 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:37.069012 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c745ebd4-ca73-4d5f-a728-57d228e10177" path="/var/lib/kubelet/pods/c745ebd4-ca73-4d5f-a728-57d228e10177/volumes" Jul 10 21:06:55.552497 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.552462 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8"] Jul 10 21:06:55.552871 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.552811 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c745ebd4-ca73-4d5f-a728-57d228e10177" containerName="manager" Jul 10 21:06:55.552871 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.552826 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c745ebd4-ca73-4d5f-a728-57d228e10177" containerName="manager" Jul 10 21:06:55.552945 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.552914 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c745ebd4-ca73-4d5f-a728-57d228e10177" containerName="manager" Jul 10 21:06:55.562479 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.562451 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" Jul 10 21:06:55.565167 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.565142 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tx9km\"" Jul 10 21:06:55.566212 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.566182 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 10 21:06:55.566615 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.566582 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8"] Jul 10 21:06:55.588018 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.587983 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k95gf\" (UniqueName: \"kubernetes.io/projected/73e43358-72b1-4db4-9d50-11a17441d1da-kube-api-access-k95gf\") pod \"maas-api-65ccc6b695-kjmw8\" (UID: \"73e43358-72b1-4db4-9d50-11a17441d1da\") " pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" Jul 10 21:06:55.588018 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.588021 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73e43358-72b1-4db4-9d50-11a17441d1da-maas-api-tls\") pod \"maas-api-65ccc6b695-kjmw8\" (UID: \"73e43358-72b1-4db4-9d50-11a17441d1da\") " pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" Jul 10 21:06:55.588222 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.588040 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73e43358-72b1-4db4-9d50-11a17441d1da-openshift-service-ca\") pod \"maas-api-65ccc6b695-kjmw8\" (UID: \"73e43358-72b1-4db4-9d50-11a17441d1da\") " pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" Jul 10 21:06:55.688716 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.688682 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k95gf\" (UniqueName: \"kubernetes.io/projected/73e43358-72b1-4db4-9d50-11a17441d1da-kube-api-access-k95gf\") pod \"maas-api-65ccc6b695-kjmw8\" (UID: \"73e43358-72b1-4db4-9d50-11a17441d1da\") " pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" Jul 10 21:06:55.688926 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.688737 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73e43358-72b1-4db4-9d50-11a17441d1da-maas-api-tls\") pod \"maas-api-65ccc6b695-kjmw8\" (UID: \"73e43358-72b1-4db4-9d50-11a17441d1da\") " pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" Jul 10 21:06:55.688926 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.688770 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73e43358-72b1-4db4-9d50-11a17441d1da-openshift-service-ca\") pod \"maas-api-65ccc6b695-kjmw8\" (UID: \"73e43358-72b1-4db4-9d50-11a17441d1da\") " pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" Jul 10 21:06:55.689523 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.689498 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73e43358-72b1-4db4-9d50-11a17441d1da-openshift-service-ca\") pod \"maas-api-65ccc6b695-kjmw8\" (UID: \"73e43358-72b1-4db4-9d50-11a17441d1da\") " pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" Jul 10 21:06:55.691288 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.691246 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73e43358-72b1-4db4-9d50-11a17441d1da-maas-api-tls\") pod \"maas-api-65ccc6b695-kjmw8\" (UID: \"73e43358-72b1-4db4-9d50-11a17441d1da\") " pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" Jul 10 21:06:55.698526 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.698503 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k95gf\" (UniqueName: \"kubernetes.io/projected/73e43358-72b1-4db4-9d50-11a17441d1da-kube-api-access-k95gf\") pod \"maas-api-65ccc6b695-kjmw8\" (UID: \"73e43358-72b1-4db4-9d50-11a17441d1da\") " pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" Jul 10 21:06:55.874367 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:55.874290 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" Jul 10 21:06:56.048327 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.048301 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8"] Jul 10 21:06:56.051265 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:06:56.051231 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e43358_72b1_4db4_9d50_11a17441d1da.slice/crio-29429aa1096608556c9ef2e65690f91be04469b84795766e35773af0cfc82e03 WatchSource:0}: Error finding container 29429aa1096608556c9ef2e65690f91be04469b84795766e35773af0cfc82e03: Status 404 returned error can't find the container with id 29429aa1096608556c9ef2e65690f91be04469b84795766e35773af0cfc82e03 Jul 10 21:06:56.109007 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.108972 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-l8l59"] Jul 10 21:06:56.114666 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.114648 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-l8l59" Jul 10 21:06:56.117085 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.117060 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 10 21:06:56.117213 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.117060 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ksl5j\"" Jul 10 21:06:56.123156 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.123132 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-l8l59"] Jul 10 21:06:56.126806 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.126745 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-rttt6"] Jul 10 21:06:56.130637 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.130615 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-rttt6" Jul 10 21:06:56.142707 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.142680 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-rttt6"] Jul 10 21:06:56.193812 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.193772 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrc8q\" (UniqueName: \"kubernetes.io/projected/2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd-kube-api-access-wrc8q\") pod \"payload-processing-58f86c7798-rttt6\" (UID: \"2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd\") " pod="openshift-ingress/payload-processing-58f86c7798-rttt6" Jul 10 21:06:56.194007 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.193831 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd-plugins-config-volume\") pod \"payload-processing-58f86c7798-rttt6\" (UID: \"2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd\") " pod="openshift-ingress/payload-processing-58f86c7798-rttt6" Jul 10 21:06:56.194007 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.193849 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf-plugins-config-volume\") pod \"payload-pre-processing-7964457696-l8l59\" (UID: \"ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf\") " pod="openshift-ingress/payload-pre-processing-7964457696-l8l59" Jul 10 21:06:56.194007 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.193938 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5nv8\" (UniqueName: \"kubernetes.io/projected/ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf-kube-api-access-g5nv8\") pod \"payload-pre-processing-7964457696-l8l59\" (UID: \"ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf\") " pod="openshift-ingress/payload-pre-processing-7964457696-l8l59" Jul 10 21:06:56.295297 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.295264 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5nv8\" (UniqueName: \"kubernetes.io/projected/ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf-kube-api-access-g5nv8\") pod \"payload-pre-processing-7964457696-l8l59\" (UID: \"ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf\") " pod="openshift-ingress/payload-pre-processing-7964457696-l8l59" Jul 10 21:06:56.295522 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.295319 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrc8q\" (UniqueName: \"kubernetes.io/projected/2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd-kube-api-access-wrc8q\") pod \"payload-processing-58f86c7798-rttt6\" (UID: \"2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd\") " pod="openshift-ingress/payload-processing-58f86c7798-rttt6" Jul 10 21:06:56.295522 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.295353 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd-plugins-config-volume\") pod \"payload-processing-58f86c7798-rttt6\" (UID: \"2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd\") " pod="openshift-ingress/payload-processing-58f86c7798-rttt6" Jul 10 21:06:56.295522 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.295369 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf-plugins-config-volume\") pod \"payload-pre-processing-7964457696-l8l59\" (UID: \"ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf\") " pod="openshift-ingress/payload-pre-processing-7964457696-l8l59" Jul 10 21:06:56.296110 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.296090 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf-plugins-config-volume\") pod \"payload-pre-processing-7964457696-l8l59\" (UID: \"ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf\") " pod="openshift-ingress/payload-pre-processing-7964457696-l8l59" Jul 10 21:06:56.296147 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.296113 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd-plugins-config-volume\") pod \"payload-processing-58f86c7798-rttt6\" (UID: \"2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd\") " pod="openshift-ingress/payload-processing-58f86c7798-rttt6" Jul 10 21:06:56.303161 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.303125 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrc8q\" (UniqueName: \"kubernetes.io/projected/2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd-kube-api-access-wrc8q\") pod \"payload-processing-58f86c7798-rttt6\" (UID: \"2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd\") " pod="openshift-ingress/payload-processing-58f86c7798-rttt6" Jul 10 21:06:56.303377 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.303355 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5nv8\" (UniqueName: \"kubernetes.io/projected/ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf-kube-api-access-g5nv8\") pod \"payload-pre-processing-7964457696-l8l59\" (UID: \"ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf\") " pod="openshift-ingress/payload-pre-processing-7964457696-l8l59" Jul 10 21:06:56.305664 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.305641 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" event={"ID":"73e43358-72b1-4db4-9d50-11a17441d1da","Type":"ContainerStarted","Data":"29429aa1096608556c9ef2e65690f91be04469b84795766e35773af0cfc82e03"} Jul 10 21:06:56.427175 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.427085 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-l8l59" Jul 10 21:06:56.442183 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.442154 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-rttt6" Jul 10 21:06:56.579338 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.579306 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-l8l59"] Jul 10 21:06:56.581679 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:06:56.581646 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef8fb89d_4eb9_44cb_86b9_f3e45863fbaf.slice/crio-bab3ff84a86f8c127916017f4a7c3443c9f0817738fcd58f37e4d9539d63b36a WatchSource:0}: Error finding container bab3ff84a86f8c127916017f4a7c3443c9f0817738fcd58f37e4d9539d63b36a: Status 404 returned error can't find the container with id bab3ff84a86f8c127916017f4a7c3443c9f0817738fcd58f37e4d9539d63b36a Jul 10 21:06:56.600085 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:56.600061 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-rttt6"] Jul 10 21:06:56.602552 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:06:56.602524 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dcc9082_e9cd_43d3_8e9c_fa64ae45a9cd.slice/crio-5ef699603fd5529dea1850e02998393d1d52b27a07975a78e4f1b84f57b1ef19 WatchSource:0}: Error finding container 5ef699603fd5529dea1850e02998393d1d52b27a07975a78e4f1b84f57b1ef19: Status 404 returned error can't find the container with id 5ef699603fd5529dea1850e02998393d1d52b27a07975a78e4f1b84f57b1ef19 Jul 10 21:06:57.315617 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:57.315578 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-l8l59" event={"ID":"ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf","Type":"ContainerStarted","Data":"bab3ff84a86f8c127916017f4a7c3443c9f0817738fcd58f37e4d9539d63b36a"} Jul 10 21:06:57.317134 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:57.317097 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-rttt6" event={"ID":"2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd","Type":"ContainerStarted","Data":"5ef699603fd5529dea1850e02998393d1d52b27a07975a78e4f1b84f57b1ef19"} Jul 10 21:06:59.331233 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:59.331196 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" event={"ID":"73e43358-72b1-4db4-9d50-11a17441d1da","Type":"ContainerStarted","Data":"1149c19c9c4ab3347a778a1b32ca34ad41635c3ccc695e1259d176640a3fe40b"} Jul 10 21:06:59.331747 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:59.331312 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" Jul 10 21:06:59.332775 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:59.332750 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-l8l59" event={"ID":"ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf","Type":"ContainerStarted","Data":"3ea60f3e711620a6b20336712a20f4393f7999f2c454f4c6073a5faf0ee27b4a"} Jul 10 21:06:59.332972 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:59.332815 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-l8l59" Jul 10 21:06:59.334145 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:59.334123 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-rttt6" event={"ID":"2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd","Type":"ContainerStarted","Data":"0ee69a7b06c8493b7a9e90773ee15d098be97f04df17a5908260638e4e0f42f5"} Jul 10 21:06:59.334243 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:59.334172 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-rttt6" Jul 10 21:06:59.348698 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:59.348647 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" podStartSLOduration=2.105438875 podStartE2EDuration="4.348631683s" totalImagesPullingTime="2.243192808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:06:59.348012079 +0000 UTC m=+552.931707708" watchObservedRunningTime="2026-07-10 21:06:59.348631683 +0000 UTC m=+552.932327286" Jul 10 21:06:59.362784 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:59.362734 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-l8l59" podStartSLOduration=1.65036423 podStartE2EDuration="3.362718218s" totalImagesPullingTime="1.712353988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:06:59.362109054 +0000 UTC m=+552.945804657" watchObservedRunningTime="2026-07-10 21:06:59.362718218 +0000 UTC m=+552.946413820" Jul 10 21:06:59.376963 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:06:59.376911 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-rttt6" podStartSLOduration=1.681961552 podStartE2EDuration="3.376891817s" totalImagesPullingTime="1.694930265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:06:59.376187621 +0000 UTC m=+552.959883227" watchObservedRunningTime="2026-07-10 21:06:59.376891817 +0000 UTC m=+552.960587422" Jul 10 21:07:05.345106 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:05.345076 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-65ccc6b695-kjmw8" Jul 10 21:07:10.341045 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:10.341005 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-rttt6" Jul 10 21:07:10.341646 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:10.341342 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-l8l59" Jul 10 21:07:23.566294 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.566262 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t"] Jul 10 21:07:23.569707 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.569684 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.573440 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.573410 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 21:07:23.573583 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.573469 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 10 21:07:23.573583 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.573500 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-phvp8\"" Jul 10 21:07:23.577413 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.574115 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 21:07:23.586583 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.586557 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t"] Jul 10 21:07:23.739910 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.739867 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b3518d-d35b-4e26-86e6-e312bfaafd92-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.739910 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.739916 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b3518d-d35b-4e26-86e6-e312bfaafd92-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.740120 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.739949 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8758\" (UniqueName: \"kubernetes.io/projected/16b3518d-d35b-4e26-86e6-e312bfaafd92-kube-api-access-h8758\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.740120 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.739983 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b3518d-d35b-4e26-86e6-e312bfaafd92-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.740120 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.739999 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b3518d-d35b-4e26-86e6-e312bfaafd92-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.740120 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.740013 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b3518d-d35b-4e26-86e6-e312bfaafd92-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.840801 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.840708 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b3518d-d35b-4e26-86e6-e312bfaafd92-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.840801 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.840768 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b3518d-d35b-4e26-86e6-e312bfaafd92-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.841016 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.840805 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8758\" (UniqueName: \"kubernetes.io/projected/16b3518d-d35b-4e26-86e6-e312bfaafd92-kube-api-access-h8758\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.841016 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.840831 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b3518d-d35b-4e26-86e6-e312bfaafd92-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.841016 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.840857 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b3518d-d35b-4e26-86e6-e312bfaafd92-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.841016 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.840880 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b3518d-d35b-4e26-86e6-e312bfaafd92-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.841241 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.841221 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b3518d-d35b-4e26-86e6-e312bfaafd92-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.841301 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.841269 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b3518d-d35b-4e26-86e6-e312bfaafd92-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.841358 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.841307 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b3518d-d35b-4e26-86e6-e312bfaafd92-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.843344 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.843326 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b3518d-d35b-4e26-86e6-e312bfaafd92-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.843442 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.843372 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b3518d-d35b-4e26-86e6-e312bfaafd92-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.848278 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.848255 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8758\" (UniqueName: \"kubernetes.io/projected/16b3518d-d35b-4e26-86e6-e312bfaafd92-kube-api-access-h8758\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rrz8t\" (UID: \"16b3518d-d35b-4e26-86e6-e312bfaafd92\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:23.882304 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:23.882265 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:24.017750 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:24.017725 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t"] Jul 10 21:07:24.019769 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:07:24.019734 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b3518d_d35b_4e26_86e6_e312bfaafd92.slice/crio-d59f6bbb273fb618486ea7458374546de51deaa4e407ee06294635c898d880bf WatchSource:0}: Error finding container d59f6bbb273fb618486ea7458374546de51deaa4e407ee06294635c898d880bf: Status 404 returned error can't find the container with id d59f6bbb273fb618486ea7458374546de51deaa4e407ee06294635c898d880bf Jul 10 21:07:24.454517 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:24.454471 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" event={"ID":"16b3518d-d35b-4e26-86e6-e312bfaafd92","Type":"ContainerStarted","Data":"d59f6bbb273fb618486ea7458374546de51deaa4e407ee06294635c898d880bf"} Jul 10 21:07:25.460517 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:25.460477 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" event={"ID":"16b3518d-d35b-4e26-86e6-e312bfaafd92","Type":"ContainerStarted","Data":"58b1edd3ba1823934547ebcd1fcbc02a583dca1d59ef8e493da0e329809bc88a"} Jul 10 21:07:25.479259 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:25.479160 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" podStartSLOduration=1.280319234 podStartE2EDuration="2.479142661s" totalImagesPullingTime="1.198823427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:07:25.477538315 +0000 UTC m=+579.061233918" watchObservedRunningTime="2026-07-10 21:07:25.479142661 +0000 UTC m=+579.062838262" Jul 10 21:07:33.882777 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:33.882738 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:33.883277 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:33.882793 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:33.899120 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:33.899083 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:07:34.514944 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:07:34.514916 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rrz8t" Jul 10 21:08:05.965685 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:05.965648 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8"] Jul 10 21:08:05.968618 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:05.968546 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:05.971103 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:05.971079 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 10 21:08:05.978070 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:05.978049 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8"] Jul 10 21:08:06.104945 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.104903 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.104945 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.104948 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.105163 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.104964 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.105163 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.104985 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tswq7\" (UniqueName: \"kubernetes.io/projected/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-kube-api-access-tswq7\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.105163 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.105028 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.105163 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.105045 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.206310 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.206271 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.206310 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.206310 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.206614 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.206412 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.206614 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.206445 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.206614 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.206467 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.206614 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.206499 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tswq7\" (UniqueName: \"kubernetes.io/projected/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-kube-api-access-tswq7\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.206985 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.206962 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.207092 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.206985 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.207215 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.207188 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.209047 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.209023 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.209277 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.209257 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.214614 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.214564 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tswq7\" (UniqueName: \"kubernetes.io/projected/0ceb18bc-3c9a-481d-acce-2219d0ed3c12-kube-api-access-tswq7\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8\" (UID: \"0ceb18bc-3c9a-481d-acce-2219d0ed3c12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.279083 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.278999 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:06.413411 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.413370 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8"] Jul 10 21:08:06.416927 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:08:06.416897 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ceb18bc_3c9a_481d_acce_2219d0ed3c12.slice/crio-d297db84771837dc67f92411f5249b2ac02e7ce288be948d82755e41c8c52aa9 WatchSource:0}: Error finding container d297db84771837dc67f92411f5249b2ac02e7ce288be948d82755e41c8c52aa9: Status 404 returned error can't find the container with id d297db84771837dc67f92411f5249b2ac02e7ce288be948d82755e41c8c52aa9 Jul 10 21:08:06.418425 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.418381 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 21:08:06.653230 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:06.653201 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" event={"ID":"0ceb18bc-3c9a-481d-acce-2219d0ed3c12","Type":"ContainerStarted","Data":"d297db84771837dc67f92411f5249b2ac02e7ce288be948d82755e41c8c52aa9"} Jul 10 21:08:07.659618 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:07.659583 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" event={"ID":"0ceb18bc-3c9a-481d-acce-2219d0ed3c12","Type":"ContainerStarted","Data":"8afd70c12a6e238ed609ef6563f610259ad74369724b738820c67f04dfeb6251"} Jul 10 21:08:07.677741 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:07.677688 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" podStartSLOduration=2.465780395 podStartE2EDuration="2.677670422s" totalImagesPullingTime="211.890027ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:08:07.676536677 +0000 UTC m=+621.260232280" watchObservedRunningTime="2026-07-10 21:08:07.677670422 +0000 UTC m=+621.261366023" Jul 10 21:08:16.279505 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:16.279470 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:16.279944 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:16.279545 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:16.292250 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:16.292222 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:08:16.712284 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:08:16.712254 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8" Jul 10 21:09:09.764513 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:09.764478 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d79d57689-gx8qv"] Jul 10 21:09:09.767004 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:09.766984 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d79d57689-gx8qv" Jul 10 21:09:09.777637 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:09.777610 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d79d57689-gx8qv"] Jul 10 21:09:09.874086 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:09.874052 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/55911c4c-7f7c-46f7-bbec-9dc59a665b49-tls-cert\") pod \"authorino-6d79d57689-gx8qv\" (UID: \"55911c4c-7f7c-46f7-bbec-9dc59a665b49\") " pod="kuadrant-system/authorino-6d79d57689-gx8qv" Jul 10 21:09:09.874267 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:09.874174 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwb7v\" (UniqueName: \"kubernetes.io/projected/55911c4c-7f7c-46f7-bbec-9dc59a665b49-kube-api-access-kwb7v\") pod \"authorino-6d79d57689-gx8qv\" (UID: \"55911c4c-7f7c-46f7-bbec-9dc59a665b49\") " pod="kuadrant-system/authorino-6d79d57689-gx8qv" Jul 10 21:09:09.975481 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:09.975445 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwb7v\" (UniqueName: \"kubernetes.io/projected/55911c4c-7f7c-46f7-bbec-9dc59a665b49-kube-api-access-kwb7v\") pod \"authorino-6d79d57689-gx8qv\" (UID: \"55911c4c-7f7c-46f7-bbec-9dc59a665b49\") " pod="kuadrant-system/authorino-6d79d57689-gx8qv" Jul 10 21:09:09.975674 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:09.975499 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/55911c4c-7f7c-46f7-bbec-9dc59a665b49-tls-cert\") pod \"authorino-6d79d57689-gx8qv\" (UID: \"55911c4c-7f7c-46f7-bbec-9dc59a665b49\") " pod="kuadrant-system/authorino-6d79d57689-gx8qv" Jul 10 21:09:09.978069 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:09.978044 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/55911c4c-7f7c-46f7-bbec-9dc59a665b49-tls-cert\") pod \"authorino-6d79d57689-gx8qv\" (UID: \"55911c4c-7f7c-46f7-bbec-9dc59a665b49\") " pod="kuadrant-system/authorino-6d79d57689-gx8qv" Jul 10 21:09:09.984319 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:09.984293 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwb7v\" (UniqueName: \"kubernetes.io/projected/55911c4c-7f7c-46f7-bbec-9dc59a665b49-kube-api-access-kwb7v\") pod \"authorino-6d79d57689-gx8qv\" (UID: \"55911c4c-7f7c-46f7-bbec-9dc59a665b49\") " pod="kuadrant-system/authorino-6d79d57689-gx8qv" Jul 10 21:09:10.076324 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:10.076239 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d79d57689-gx8qv" Jul 10 21:09:10.213827 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:10.213792 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d79d57689-gx8qv"] Jul 10 21:09:10.214892 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:09:10.214865 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55911c4c_7f7c_46f7_bbec_9dc59a665b49.slice/crio-a4aad28e8bbffe3551399b15cfde75695697a76e07d78cda10f3c4c99fba1c93 WatchSource:0}: Error finding container a4aad28e8bbffe3551399b15cfde75695697a76e07d78cda10f3c4c99fba1c93: Status 404 returned error can't find the container with id a4aad28e8bbffe3551399b15cfde75695697a76e07d78cda10f3c4c99fba1c93 Jul 10 21:09:10.955041 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:10.955002 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d79d57689-gx8qv" event={"ID":"55911c4c-7f7c-46f7-bbec-9dc59a665b49","Type":"ContainerStarted","Data":"a4aad28e8bbffe3551399b15cfde75695697a76e07d78cda10f3c4c99fba1c93"} Jul 10 21:09:11.962440 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:11.962382 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d79d57689-gx8qv" event={"ID":"55911c4c-7f7c-46f7-bbec-9dc59a665b49","Type":"ContainerStarted","Data":"7b34e3eb3856719a241791a392989440abb51f64cf5c21ceeb0db4ff094bc894"} Jul 10 21:09:11.983774 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:11.983708 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d79d57689-gx8qv" podStartSLOduration=2.033264944 podStartE2EDuration="2.98368538s" totalImagesPullingTime="950.420436ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:09:11.982097682 +0000 UTC m=+685.565793295" watchObservedRunningTime="2026-07-10 21:09:11.98368538 +0000 UTC m=+685.567380983" Jul 10 21:09:12.011760 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.011717 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65c8788bc6-9hnrs"] Jul 10 21:09:12.012126 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.011995 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-65c8788bc6-9hnrs" podUID="b98e476d-c456-4eac-9dd5-48c2079aabcc" containerName="authorino" containerID="cri-o://391fae72117a7f1b1a81fb26faf2994eea4a51b1222ec61dbf04343be1d33d51" gracePeriod=30 Jul 10 21:09:12.278910 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.278878 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65c8788bc6-9hnrs" Jul 10 21:09:12.405165 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.405132 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdtz6\" (UniqueName: \"kubernetes.io/projected/b98e476d-c456-4eac-9dd5-48c2079aabcc-kube-api-access-pdtz6\") pod \"b98e476d-c456-4eac-9dd5-48c2079aabcc\" (UID: \"b98e476d-c456-4eac-9dd5-48c2079aabcc\") " Jul 10 21:09:12.405326 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.405188 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b98e476d-c456-4eac-9dd5-48c2079aabcc-tls-cert\") pod \"b98e476d-c456-4eac-9dd5-48c2079aabcc\" (UID: \"b98e476d-c456-4eac-9dd5-48c2079aabcc\") " Jul 10 21:09:12.407306 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.407267 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b98e476d-c456-4eac-9dd5-48c2079aabcc-kube-api-access-pdtz6" (OuterVolumeSpecName: "kube-api-access-pdtz6") pod "b98e476d-c456-4eac-9dd5-48c2079aabcc" (UID: "b98e476d-c456-4eac-9dd5-48c2079aabcc"). InnerVolumeSpecName "kube-api-access-pdtz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:09:12.418738 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.418697 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b98e476d-c456-4eac-9dd5-48c2079aabcc-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "b98e476d-c456-4eac-9dd5-48c2079aabcc" (UID: "b98e476d-c456-4eac-9dd5-48c2079aabcc"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 21:09:12.506142 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.506103 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdtz6\" (UniqueName: \"kubernetes.io/projected/b98e476d-c456-4eac-9dd5-48c2079aabcc-kube-api-access-pdtz6\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:09:12.506142 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.506136 2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b98e476d-c456-4eac-9dd5-48c2079aabcc-tls-cert\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:09:12.968632 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.968600 2566 generic.go:356] "Generic (PLEG): container finished" podID="b98e476d-c456-4eac-9dd5-48c2079aabcc" containerID="391fae72117a7f1b1a81fb26faf2994eea4a51b1222ec61dbf04343be1d33d51" exitCode=0 Jul 10 21:09:12.969135 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.968643 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65c8788bc6-9hnrs" Jul 10 21:09:12.969135 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.968692 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65c8788bc6-9hnrs" event={"ID":"b98e476d-c456-4eac-9dd5-48c2079aabcc","Type":"ContainerDied","Data":"391fae72117a7f1b1a81fb26faf2994eea4a51b1222ec61dbf04343be1d33d51"} Jul 10 21:09:12.969135 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.968738 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65c8788bc6-9hnrs" event={"ID":"b98e476d-c456-4eac-9dd5-48c2079aabcc","Type":"ContainerDied","Data":"7e2cfb5288b2a72dd1d2a00d09a2303954417b3a0fc0663487ff782abda10854"} Jul 10 21:09:12.969135 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.968764 2566 scope.go:117] "RemoveContainer" containerID="391fae72117a7f1b1a81fb26faf2994eea4a51b1222ec61dbf04343be1d33d51" Jul 10 21:09:12.979302 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.979282 2566 scope.go:117] "RemoveContainer" containerID="391fae72117a7f1b1a81fb26faf2994eea4a51b1222ec61dbf04343be1d33d51" Jul 10 21:09:12.979628 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:09:12.979603 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"391fae72117a7f1b1a81fb26faf2994eea4a51b1222ec61dbf04343be1d33d51\": container with ID starting with 391fae72117a7f1b1a81fb26faf2994eea4a51b1222ec61dbf04343be1d33d51 not found: ID does not exist" containerID="391fae72117a7f1b1a81fb26faf2994eea4a51b1222ec61dbf04343be1d33d51" Jul 10 21:09:12.979726 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.979635 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"391fae72117a7f1b1a81fb26faf2994eea4a51b1222ec61dbf04343be1d33d51"} err="failed to get container status \"391fae72117a7f1b1a81fb26faf2994eea4a51b1222ec61dbf04343be1d33d51\": rpc error: code = NotFound desc = could not find container \"391fae72117a7f1b1a81fb26faf2994eea4a51b1222ec61dbf04343be1d33d51\": container with ID starting with 391fae72117a7f1b1a81fb26faf2994eea4a51b1222ec61dbf04343be1d33d51 not found: ID does not exist" Jul 10 21:09:12.989843 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.989819 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-65c8788bc6-9hnrs"] Jul 10 21:09:12.997548 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:12.997526 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-65c8788bc6-9hnrs"] Jul 10 21:09:13.069244 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:09:13.069209 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b98e476d-c456-4eac-9dd5-48c2079aabcc" path="/var/lib/kubelet/pods/b98e476d-c456-4eac-9dd5-48c2079aabcc/volumes" Jul 10 21:10:23.356734 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:23.356695 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5db9df9f4b-25w8g"] Jul 10 21:10:23.357272 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:23.356895 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5db9df9f4b-25w8g" podUID="c7cc230d-ec39-4361-9cec-a5866709e755" containerName="manager" containerID="cri-o://2bcf0a057a91fb4adfe24dee8a959f95e253cd09030d4ea65fecadc65b90108a" gracePeriod=30 Jul 10 21:10:23.611621 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:23.611543 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5db9df9f4b-25w8g" Jul 10 21:10:23.728444 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:23.728406 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv86j\" (UniqueName: \"kubernetes.io/projected/c7cc230d-ec39-4361-9cec-a5866709e755-kube-api-access-nv86j\") pod \"c7cc230d-ec39-4361-9cec-a5866709e755\" (UID: \"c7cc230d-ec39-4361-9cec-a5866709e755\") " Jul 10 21:10:23.728620 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:23.728533 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7cc230d-ec39-4361-9cec-a5866709e755-webhook-cert\") pod \"c7cc230d-ec39-4361-9cec-a5866709e755\" (UID: \"c7cc230d-ec39-4361-9cec-a5866709e755\") " Jul 10 21:10:23.730561 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:23.730530 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7cc230d-ec39-4361-9cec-a5866709e755-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c7cc230d-ec39-4361-9cec-a5866709e755" (UID: "c7cc230d-ec39-4361-9cec-a5866709e755"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 21:10:23.730712 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:23.730584 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7cc230d-ec39-4361-9cec-a5866709e755-kube-api-access-nv86j" (OuterVolumeSpecName: "kube-api-access-nv86j") pod "c7cc230d-ec39-4361-9cec-a5866709e755" (UID: "c7cc230d-ec39-4361-9cec-a5866709e755"). InnerVolumeSpecName "kube-api-access-nv86j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:10:23.830130 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:23.830093 2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7cc230d-ec39-4361-9cec-a5866709e755-webhook-cert\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:10:23.830130 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:23.830123 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv86j\" (UniqueName: \"kubernetes.io/projected/c7cc230d-ec39-4361-9cec-a5866709e755-kube-api-access-nv86j\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:10:24.308866 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:24.308821 2566 generic.go:356] "Generic (PLEG): container finished" podID="c7cc230d-ec39-4361-9cec-a5866709e755" containerID="2bcf0a057a91fb4adfe24dee8a959f95e253cd09030d4ea65fecadc65b90108a" exitCode=0 Jul 10 21:10:24.308866 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:24.308879 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5db9df9f4b-25w8g" event={"ID":"c7cc230d-ec39-4361-9cec-a5866709e755","Type":"ContainerDied","Data":"2bcf0a057a91fb4adfe24dee8a959f95e253cd09030d4ea65fecadc65b90108a"} Jul 10 21:10:24.309094 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:24.308900 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5db9df9f4b-25w8g" Jul 10 21:10:24.309094 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:24.308961 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5db9df9f4b-25w8g" event={"ID":"c7cc230d-ec39-4361-9cec-a5866709e755","Type":"ContainerDied","Data":"f1e42f861eb9462cfba027e060a2a739f89cbc32e0bf81ac84cc936a89ce2991"} Jul 10 21:10:24.309094 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:24.308991 2566 scope.go:117] "RemoveContainer" containerID="2bcf0a057a91fb4adfe24dee8a959f95e253cd09030d4ea65fecadc65b90108a" Jul 10 21:10:24.322372 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:24.322346 2566 scope.go:117] "RemoveContainer" containerID="2bcf0a057a91fb4adfe24dee8a959f95e253cd09030d4ea65fecadc65b90108a" Jul 10 21:10:24.322791 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:10:24.322769 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bcf0a057a91fb4adfe24dee8a959f95e253cd09030d4ea65fecadc65b90108a\": container with ID starting with 2bcf0a057a91fb4adfe24dee8a959f95e253cd09030d4ea65fecadc65b90108a not found: ID does not exist" containerID="2bcf0a057a91fb4adfe24dee8a959f95e253cd09030d4ea65fecadc65b90108a" Jul 10 21:10:24.322864 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:24.322799 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bcf0a057a91fb4adfe24dee8a959f95e253cd09030d4ea65fecadc65b90108a"} err="failed to get container status \"2bcf0a057a91fb4adfe24dee8a959f95e253cd09030d4ea65fecadc65b90108a\": rpc error: code = NotFound desc = could not find container \"2bcf0a057a91fb4adfe24dee8a959f95e253cd09030d4ea65fecadc65b90108a\": container with ID starting with 2bcf0a057a91fb4adfe24dee8a959f95e253cd09030d4ea65fecadc65b90108a not found: ID does not exist" Jul 10 21:10:24.333576 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:24.333543 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5db9df9f4b-25w8g"] Jul 10 21:10:24.337421 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:24.337368 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5db9df9f4b-25w8g"] Jul 10 21:10:25.069770 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.069739 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7cc230d-ec39-4361-9cec-a5866709e755" path="/var/lib/kubelet/pods/c7cc230d-ec39-4361-9cec-a5866709e755/volumes" Jul 10 21:10:25.070166 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.070006 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5db9df9f4b-zmzfd"] Jul 10 21:10:25.070325 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.070313 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b98e476d-c456-4eac-9dd5-48c2079aabcc" containerName="authorino" Jul 10 21:10:25.070360 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.070327 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b98e476d-c456-4eac-9dd5-48c2079aabcc" containerName="authorino" Jul 10 21:10:25.070360 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.070350 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7cc230d-ec39-4361-9cec-a5866709e755" containerName="manager" Jul 10 21:10:25.070360 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.070356 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7cc230d-ec39-4361-9cec-a5866709e755" containerName="manager" Jul 10 21:10:25.070480 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.070432 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7cc230d-ec39-4361-9cec-a5866709e755" containerName="manager" Jul 10 21:10:25.070480 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.070443 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b98e476d-c456-4eac-9dd5-48c2079aabcc" containerName="authorino" Jul 10 21:10:25.072969 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.072953 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5db9df9f4b-zmzfd" Jul 10 21:10:25.076093 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.076059 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sfcct\"" Jul 10 21:10:25.076093 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.076063 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 10 21:10:25.076093 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.076092 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 21:10:25.079614 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.079594 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5db9df9f4b-zmzfd"] Jul 10 21:10:25.142576 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.142534 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xctlf\" (UniqueName: \"kubernetes.io/projected/41b86611-0dcc-48e9-8c31-4419ea71b58c-kube-api-access-xctlf\") pod \"maas-controller-5db9df9f4b-zmzfd\" (UID: \"41b86611-0dcc-48e9-8c31-4419ea71b58c\") " pod="opendatahub/maas-controller-5db9df9f4b-zmzfd" Jul 10 21:10:25.142802 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.142635 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41b86611-0dcc-48e9-8c31-4419ea71b58c-webhook-cert\") pod \"maas-controller-5db9df9f4b-zmzfd\" (UID: \"41b86611-0dcc-48e9-8c31-4419ea71b58c\") " pod="opendatahub/maas-controller-5db9df9f4b-zmzfd" Jul 10 21:10:25.243848 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.243742 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41b86611-0dcc-48e9-8c31-4419ea71b58c-webhook-cert\") pod \"maas-controller-5db9df9f4b-zmzfd\" (UID: \"41b86611-0dcc-48e9-8c31-4419ea71b58c\") " pod="opendatahub/maas-controller-5db9df9f4b-zmzfd" Jul 10 21:10:25.243848 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.243843 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xctlf\" (UniqueName: \"kubernetes.io/projected/41b86611-0dcc-48e9-8c31-4419ea71b58c-kube-api-access-xctlf\") pod \"maas-controller-5db9df9f4b-zmzfd\" (UID: \"41b86611-0dcc-48e9-8c31-4419ea71b58c\") " pod="opendatahub/maas-controller-5db9df9f4b-zmzfd" Jul 10 21:10:25.246224 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.246200 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41b86611-0dcc-48e9-8c31-4419ea71b58c-webhook-cert\") pod \"maas-controller-5db9df9f4b-zmzfd\" (UID: \"41b86611-0dcc-48e9-8c31-4419ea71b58c\") " pod="opendatahub/maas-controller-5db9df9f4b-zmzfd" Jul 10 21:10:25.252822 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.252799 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xctlf\" (UniqueName: \"kubernetes.io/projected/41b86611-0dcc-48e9-8c31-4419ea71b58c-kube-api-access-xctlf\") pod \"maas-controller-5db9df9f4b-zmzfd\" (UID: \"41b86611-0dcc-48e9-8c31-4419ea71b58c\") " pod="opendatahub/maas-controller-5db9df9f4b-zmzfd" Jul 10 21:10:25.384713 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.384677 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5db9df9f4b-zmzfd" Jul 10 21:10:25.529685 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:25.529657 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5db9df9f4b-zmzfd"] Jul 10 21:10:25.531212 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:10:25.531186 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41b86611_0dcc_48e9_8c31_4419ea71b58c.slice/crio-db05eb2934b25a0236d901517e90d3d0296202995ad6bd1c6ad734eedc9db7ed WatchSource:0}: Error finding container db05eb2934b25a0236d901517e90d3d0296202995ad6bd1c6ad734eedc9db7ed: Status 404 returned error can't find the container with id db05eb2934b25a0236d901517e90d3d0296202995ad6bd1c6ad734eedc9db7ed Jul 10 21:10:26.322085 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:26.322047 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5db9df9f4b-zmzfd" event={"ID":"41b86611-0dcc-48e9-8c31-4419ea71b58c","Type":"ContainerStarted","Data":"802c8922f760e400c756d479c36c93adc189fbbad1715a23c1908fac2cce5e46"} Jul 10 21:10:26.322504 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:26.322112 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5db9df9f4b-zmzfd" event={"ID":"41b86611-0dcc-48e9-8c31-4419ea71b58c","Type":"ContainerStarted","Data":"db05eb2934b25a0236d901517e90d3d0296202995ad6bd1c6ad734eedc9db7ed"} Jul 10 21:10:26.322504 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:26.322157 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5db9df9f4b-zmzfd" Jul 10 21:10:26.340499 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:26.340444 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5db9df9f4b-zmzfd" podStartSLOduration=0.954409135 podStartE2EDuration="1.340426188s" totalImagesPullingTime="386.017053ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:10:26.339034848 +0000 UTC m=+759.922730452" watchObservedRunningTime="2026-07-10 21:10:26.340426188 +0000 UTC m=+759.924121789" Jul 10 21:10:37.333542 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:10:37.333510 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5db9df9f4b-zmzfd" Jul 10 21:15:31.882419 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:31.882333 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks"] Jul 10 21:15:31.883053 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:31.882626 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" podUID="2a17d474-5464-4ed5-ac83-f269ef2d50d3" containerName="manager" containerID="cri-o://b9e64e6ebe48e15680db50277180ee3d3f338ee9946019079d8f506f400d4f31" gracePeriod=10 Jul 10 21:15:35.765704 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:35.765675 2566 generic.go:356] "Generic (PLEG): container finished" podID="2a17d474-5464-4ed5-ac83-f269ef2d50d3" containerID="b9e64e6ebe48e15680db50277180ee3d3f338ee9946019079d8f506f400d4f31" exitCode=0 Jul 10 21:15:35.766036 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:35.765738 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" event={"ID":"2a17d474-5464-4ed5-ac83-f269ef2d50d3","Type":"ContainerDied","Data":"b9e64e6ebe48e15680db50277180ee3d3f338ee9946019079d8f506f400d4f31"} Jul 10 21:15:35.829671 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:35.829647 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" Jul 10 21:15:35.920597 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:35.920566 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2k79\" (UniqueName: \"kubernetes.io/projected/2a17d474-5464-4ed5-ac83-f269ef2d50d3-kube-api-access-r2k79\") pod \"2a17d474-5464-4ed5-ac83-f269ef2d50d3\" (UID: \"2a17d474-5464-4ed5-ac83-f269ef2d50d3\") " Jul 10 21:15:35.920754 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:35.920671 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a17d474-5464-4ed5-ac83-f269ef2d50d3-extensions-socket-volume\") pod \"2a17d474-5464-4ed5-ac83-f269ef2d50d3\" (UID: \"2a17d474-5464-4ed5-ac83-f269ef2d50d3\") " Jul 10 21:15:35.921053 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:35.921032 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a17d474-5464-4ed5-ac83-f269ef2d50d3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2a17d474-5464-4ed5-ac83-f269ef2d50d3" (UID: "2a17d474-5464-4ed5-ac83-f269ef2d50d3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:15:35.922545 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:35.922523 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a17d474-5464-4ed5-ac83-f269ef2d50d3-kube-api-access-r2k79" (OuterVolumeSpecName: "kube-api-access-r2k79") pod "2a17d474-5464-4ed5-ac83-f269ef2d50d3" (UID: "2a17d474-5464-4ed5-ac83-f269ef2d50d3"). InnerVolumeSpecName "kube-api-access-r2k79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:15:36.021850 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:36.021809 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a17d474-5464-4ed5-ac83-f269ef2d50d3-extensions-socket-volume\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:15:36.021850 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:36.021840 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2k79\" (UniqueName: \"kubernetes.io/projected/2a17d474-5464-4ed5-ac83-f269ef2d50d3-kube-api-access-r2k79\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:15:36.772277 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:36.772240 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" event={"ID":"2a17d474-5464-4ed5-ac83-f269ef2d50d3","Type":"ContainerDied","Data":"19b12dc950e68a56b240f76ad68c74f4c54ad6c5968ff267fd0e15ab78fdfa85"} Jul 10 21:15:36.772705 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:36.772303 2566 scope.go:117] "RemoveContainer" containerID="b9e64e6ebe48e15680db50277180ee3d3f338ee9946019079d8f506f400d4f31" Jul 10 21:15:36.772705 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:36.772250 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks" Jul 10 21:15:36.798851 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:36.798829 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks"] Jul 10 21:15:36.807100 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:36.807071 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ldlks"] Jul 10 21:15:37.067905 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:15:37.067826 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a17d474-5464-4ed5-ac83-f269ef2d50d3" path="/var/lib/kubelet/pods/2a17d474-5464-4ed5-ac83-f269ef2d50d3/volumes" Jul 10 21:16:33.000598 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.000560 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f"] Jul 10 21:16:33.001075 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.000928 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a17d474-5464-4ed5-ac83-f269ef2d50d3" containerName="manager" Jul 10 21:16:33.001075 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.000939 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a17d474-5464-4ed5-ac83-f269ef2d50d3" containerName="manager" Jul 10 21:16:33.001075 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.001017 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a17d474-5464-4ed5-ac83-f269ef2d50d3" containerName="manager" Jul 10 21:16:33.004205 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.004186 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f" Jul 10 21:16:33.007375 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.007354 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-p7gsq\"" Jul 10 21:16:33.020246 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.020214 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f"] Jul 10 21:16:33.108797 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.108768 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81314945-8ce0-4c51-aeb7-4eac6db2b31b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zpf7f\" (UID: \"81314945-8ce0-4c51-aeb7-4eac6db2b31b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f" Jul 10 21:16:33.108971 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.108805 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77s2l\" (UniqueName: \"kubernetes.io/projected/81314945-8ce0-4c51-aeb7-4eac6db2b31b-kube-api-access-77s2l\") pod \"kuadrant-operator-controller-manager-86878c769f-zpf7f\" (UID: \"81314945-8ce0-4c51-aeb7-4eac6db2b31b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f" Jul 10 21:16:33.209763 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.209728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81314945-8ce0-4c51-aeb7-4eac6db2b31b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zpf7f\" (UID: \"81314945-8ce0-4c51-aeb7-4eac6db2b31b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f" Jul 10 21:16:33.209930 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.209781 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77s2l\" (UniqueName: \"kubernetes.io/projected/81314945-8ce0-4c51-aeb7-4eac6db2b31b-kube-api-access-77s2l\") pod \"kuadrant-operator-controller-manager-86878c769f-zpf7f\" (UID: \"81314945-8ce0-4c51-aeb7-4eac6db2b31b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f" Jul 10 21:16:33.210182 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.210149 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81314945-8ce0-4c51-aeb7-4eac6db2b31b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zpf7f\" (UID: \"81314945-8ce0-4c51-aeb7-4eac6db2b31b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f" Jul 10 21:16:33.218755 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.218730 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77s2l\" (UniqueName: \"kubernetes.io/projected/81314945-8ce0-4c51-aeb7-4eac6db2b31b-kube-api-access-77s2l\") pod \"kuadrant-operator-controller-manager-86878c769f-zpf7f\" (UID: \"81314945-8ce0-4c51-aeb7-4eac6db2b31b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f" Jul 10 21:16:33.314440 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.314320 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f" Jul 10 21:16:33.458462 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:33.458435 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f"] Jul 10 21:16:33.460744 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:16:33.460719 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81314945_8ce0_4c51_aeb7_4eac6db2b31b.slice/crio-1f80e82de7c399b8538478bd64e7cb48078f2af66c58af3fe0904934411202f5 WatchSource:0}: Error finding container 1f80e82de7c399b8538478bd64e7cb48078f2af66c58af3fe0904934411202f5: Status 404 returned error can't find the container with id 1f80e82de7c399b8538478bd64e7cb48078f2af66c58af3fe0904934411202f5 Jul 10 21:16:34.038832 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:34.038788 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f" event={"ID":"81314945-8ce0-4c51-aeb7-4eac6db2b31b","Type":"ContainerStarted","Data":"54328684b55635ee6d597328bd4b660d3e61b911e208f753bc2857c702641273"} Jul 10 21:16:34.039246 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:34.038853 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f" event={"ID":"81314945-8ce0-4c51-aeb7-4eac6db2b31b","Type":"ContainerStarted","Data":"1f80e82de7c399b8538478bd64e7cb48078f2af66c58af3fe0904934411202f5"} Jul 10 21:16:34.039246 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:34.038965 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f" Jul 10 21:16:34.064553 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:34.064483 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f" podStartSLOduration=2.064466144 podStartE2EDuration="2.064466144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:16:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:16:34.061970123 +0000 UTC m=+1127.645665725" watchObservedRunningTime="2026-07-10 21:16:34.064466144 +0000 UTC m=+1127.648161746" Jul 10 21:16:45.046905 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:16:45.046869 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zpf7f" Jul 10 21:29:21.081578 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.081549 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27"] Jul 10 21:29:21.084847 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.084826 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" Jul 10 21:29:21.087191 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.087167 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ad1d1c21-serving-cert\"" Jul 10 21:29:21.094009 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.093985 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27"] Jul 10 21:29:21.195225 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.195182 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/179be2fd-43a6-4259-a4a4-f5da1a9432aa-openshift-service-ca\") pod \"maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27\" (UID: \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" Jul 10 21:29:21.195225 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.195231 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn9m9\" (UniqueName: \"kubernetes.io/projected/179be2fd-43a6-4259-a4a4-f5da1a9432aa-kube-api-access-xn9m9\") pod \"maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27\" (UID: \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" Jul 10 21:29:21.195482 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.195338 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/179be2fd-43a6-4259-a4a4-f5da1a9432aa-maas-api-tls\") pod \"maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27\" (UID: \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" Jul 10 21:29:21.296227 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.296189 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/179be2fd-43a6-4259-a4a4-f5da1a9432aa-maas-api-tls\") pod \"maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27\" (UID: \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" Jul 10 21:29:21.296506 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.296250 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/179be2fd-43a6-4259-a4a4-f5da1a9432aa-openshift-service-ca\") pod \"maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27\" (UID: \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" Jul 10 21:29:21.296506 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.296274 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn9m9\" (UniqueName: \"kubernetes.io/projected/179be2fd-43a6-4259-a4a4-f5da1a9432aa-kube-api-access-xn9m9\") pod \"maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27\" (UID: \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" Jul 10 21:29:21.297013 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.296987 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/179be2fd-43a6-4259-a4a4-f5da1a9432aa-openshift-service-ca\") pod \"maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27\" (UID: \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" Jul 10 21:29:21.298563 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.298538 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/179be2fd-43a6-4259-a4a4-f5da1a9432aa-maas-api-tls\") pod \"maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27\" (UID: \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" Jul 10 21:29:21.305200 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.305173 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn9m9\" (UniqueName: \"kubernetes.io/projected/179be2fd-43a6-4259-a4a4-f5da1a9432aa-kube-api-access-xn9m9\") pod \"maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27\" (UID: \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" Jul 10 21:29:21.396328 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.396244 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" Jul 10 21:29:21.542563 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.542529 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27"] Jul 10 21:29:21.545289 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:29:21.545260 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod179be2fd_43a6_4259_a4a4_f5da1a9432aa.slice/crio-0ad7adf83248986f8a9b66b8785eef1619ba75526db200cca90ef371f6742bd5 WatchSource:0}: Error finding container 0ad7adf83248986f8a9b66b8785eef1619ba75526db200cca90ef371f6742bd5: Status 404 returned error can't find the container with id 0ad7adf83248986f8a9b66b8785eef1619ba75526db200cca90ef371f6742bd5 Jul 10 21:29:21.546593 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.546574 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 21:29:21.566927 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:21.566896 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" event={"ID":"179be2fd-43a6-4259-a4a4-f5da1a9432aa","Type":"ContainerStarted","Data":"0ad7adf83248986f8a9b66b8785eef1619ba75526db200cca90ef371f6742bd5"} Jul 10 21:29:22.573472 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:22.573389 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" event={"ID":"179be2fd-43a6-4259-a4a4-f5da1a9432aa","Type":"ContainerStarted","Data":"8a3d9a4d4a00528321aa64effa21820dcba5bccecb571f45aca5443ff56b1df1"} Jul 10 21:29:22.573933 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:22.573597 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" Jul 10 21:29:22.590013 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:22.589955 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" podStartSLOduration=1.143738011 podStartE2EDuration="1.589939099s" totalImagesPullingTime="446.201088ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:29:22.589382043 +0000 UTC m=+1896.173077646" watchObservedRunningTime="2026-07-10 21:29:22.589939099 +0000 UTC m=+1896.173635114" Jul 10 21:29:25.580367 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:25.580327 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27"] Jul 10 21:29:25.581165 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:25.581132 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" podUID="179be2fd-43a6-4259-a4a4-f5da1a9432aa" containerName="maas-api" containerID="cri-o://8a3d9a4d4a00528321aa64effa21820dcba5bccecb571f45aca5443ff56b1df1" gracePeriod=30 Jul 10 21:29:25.835502 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:25.835446 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" Jul 10 21:29:25.939088 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:25.939054 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/179be2fd-43a6-4259-a4a4-f5da1a9432aa-maas-api-tls\") pod \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\" (UID: \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\") " Jul 10 21:29:25.939252 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:25.939101 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xn9m9\" (UniqueName: \"kubernetes.io/projected/179be2fd-43a6-4259-a4a4-f5da1a9432aa-kube-api-access-xn9m9\") pod \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\" (UID: \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\") " Jul 10 21:29:25.939252 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:25.939150 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/179be2fd-43a6-4259-a4a4-f5da1a9432aa-openshift-service-ca\") pod \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\" (UID: \"179be2fd-43a6-4259-a4a4-f5da1a9432aa\") " Jul 10 21:29:25.939627 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:25.939600 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/179be2fd-43a6-4259-a4a4-f5da1a9432aa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "179be2fd-43a6-4259-a4a4-f5da1a9432aa" (UID: "179be2fd-43a6-4259-a4a4-f5da1a9432aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:29:25.941275 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:25.941242 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/179be2fd-43a6-4259-a4a4-f5da1a9432aa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "179be2fd-43a6-4259-a4a4-f5da1a9432aa" (UID: "179be2fd-43a6-4259-a4a4-f5da1a9432aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 21:29:25.941275 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:25.941268 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/179be2fd-43a6-4259-a4a4-f5da1a9432aa-kube-api-access-xn9m9" (OuterVolumeSpecName: "kube-api-access-xn9m9") pod "179be2fd-43a6-4259-a4a4-f5da1a9432aa" (UID: "179be2fd-43a6-4259-a4a4-f5da1a9432aa"). InnerVolumeSpecName "kube-api-access-xn9m9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:29:26.040116 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:26.040086 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/179be2fd-43a6-4259-a4a4-f5da1a9432aa-maas-api-tls\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:29:26.040116 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:26.040110 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn9m9\" (UniqueName: \"kubernetes.io/projected/179be2fd-43a6-4259-a4a4-f5da1a9432aa-kube-api-access-xn9m9\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:29:26.040116 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:26.040120 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/179be2fd-43a6-4259-a4a4-f5da1a9432aa-openshift-service-ca\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:29:26.594297 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:26.594257 2566 generic.go:356] "Generic (PLEG): container finished" podID="179be2fd-43a6-4259-a4a4-f5da1a9432aa" containerID="8a3d9a4d4a00528321aa64effa21820dcba5bccecb571f45aca5443ff56b1df1" exitCode=0 Jul 10 21:29:26.594798 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:26.594315 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" Jul 10 21:29:26.594798 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:26.594322 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" event={"ID":"179be2fd-43a6-4259-a4a4-f5da1a9432aa","Type":"ContainerDied","Data":"8a3d9a4d4a00528321aa64effa21820dcba5bccecb571f45aca5443ff56b1df1"} Jul 10 21:29:26.594798 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:26.594456 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27" event={"ID":"179be2fd-43a6-4259-a4a4-f5da1a9432aa","Type":"ContainerDied","Data":"0ad7adf83248986f8a9b66b8785eef1619ba75526db200cca90ef371f6742bd5"} Jul 10 21:29:26.594798 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:26.594485 2566 scope.go:117] "RemoveContainer" containerID="8a3d9a4d4a00528321aa64effa21820dcba5bccecb571f45aca5443ff56b1df1" Jul 10 21:29:26.604255 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:26.604234 2566 scope.go:117] "RemoveContainer" containerID="8a3d9a4d4a00528321aa64effa21820dcba5bccecb571f45aca5443ff56b1df1" Jul 10 21:29:26.604601 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:29:26.604580 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a3d9a4d4a00528321aa64effa21820dcba5bccecb571f45aca5443ff56b1df1\": container with ID starting with 8a3d9a4d4a00528321aa64effa21820dcba5bccecb571f45aca5443ff56b1df1 not found: ID does not exist" containerID="8a3d9a4d4a00528321aa64effa21820dcba5bccecb571f45aca5443ff56b1df1" Jul 10 21:29:26.604694 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:26.604608 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a3d9a4d4a00528321aa64effa21820dcba5bccecb571f45aca5443ff56b1df1"} err="failed to get container status \"8a3d9a4d4a00528321aa64effa21820dcba5bccecb571f45aca5443ff56b1df1\": rpc error: code = NotFound desc = could not find container \"8a3d9a4d4a00528321aa64effa21820dcba5bccecb571f45aca5443ff56b1df1\": container with ID starting with 8a3d9a4d4a00528321aa64effa21820dcba5bccecb571f45aca5443ff56b1df1 not found: ID does not exist" Jul 10 21:29:26.617097 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:26.617076 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27"] Jul 10 21:29:26.623961 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:26.623941 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad1d1c21-5495685cdf-zkp27"] Jul 10 21:29:27.068869 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:27.068837 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="179be2fd-43a6-4259-a4a4-f5da1a9432aa" path="/var/lib/kubelet/pods/179be2fd-43a6-4259-a4a4-f5da1a9432aa/volumes" Jul 10 21:29:33.952723 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:33.952688 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk"] Jul 10 21:29:33.953283 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:33.953262 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="179be2fd-43a6-4259-a4a4-f5da1a9432aa" containerName="maas-api" Jul 10 21:29:33.953497 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:33.953285 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="179be2fd-43a6-4259-a4a4-f5da1a9432aa" containerName="maas-api" Jul 10 21:29:33.953497 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:33.953476 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="179be2fd-43a6-4259-a4a4-f5da1a9432aa" containerName="maas-api" Jul 10 21:29:33.957869 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:33.957846 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:33.961169 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:33.961146 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-78079c65-serving-cert\"" Jul 10 21:29:33.967750 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:33.967714 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk"] Jul 10 21:29:34.109560 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:34.109514 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f850214-3252-4e08-9f9d-dedca82385a6-maas-api-tls\") pod \"maas-api-e2e-derive-78079c65-75bc99fc76-42jvk\" (UID: \"9f850214-3252-4e08-9f9d-dedca82385a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:34.109760 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:34.109641 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2txd\" (UniqueName: \"kubernetes.io/projected/9f850214-3252-4e08-9f9d-dedca82385a6-kube-api-access-l2txd\") pod \"maas-api-e2e-derive-78079c65-75bc99fc76-42jvk\" (UID: \"9f850214-3252-4e08-9f9d-dedca82385a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:34.109760 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:34.109675 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f850214-3252-4e08-9f9d-dedca82385a6-openshift-service-ca\") pod \"maas-api-e2e-derive-78079c65-75bc99fc76-42jvk\" (UID: \"9f850214-3252-4e08-9f9d-dedca82385a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:34.210786 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:34.210746 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f850214-3252-4e08-9f9d-dedca82385a6-maas-api-tls\") pod \"maas-api-e2e-derive-78079c65-75bc99fc76-42jvk\" (UID: \"9f850214-3252-4e08-9f9d-dedca82385a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:34.210948 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:34.210840 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2txd\" (UniqueName: \"kubernetes.io/projected/9f850214-3252-4e08-9f9d-dedca82385a6-kube-api-access-l2txd\") pod \"maas-api-e2e-derive-78079c65-75bc99fc76-42jvk\" (UID: \"9f850214-3252-4e08-9f9d-dedca82385a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:34.210948 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:34.210871 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f850214-3252-4e08-9f9d-dedca82385a6-openshift-service-ca\") pod \"maas-api-e2e-derive-78079c65-75bc99fc76-42jvk\" (UID: \"9f850214-3252-4e08-9f9d-dedca82385a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:34.210948 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:29:34.210915 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-serving-cert: secret "maas-api-e2e-derive-78079c65-serving-cert" not found Jul 10 21:29:34.211072 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:29:34.211004 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f850214-3252-4e08-9f9d-dedca82385a6-maas-api-tls podName:9f850214-3252-4e08-9f9d-dedca82385a6 nodeName:}" failed. No retries permitted until 2026-07-10 21:29:34.71098491 +0000 UTC m=+1908.294680490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9f850214-3252-4e08-9f9d-dedca82385a6-maas-api-tls") pod "maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" (UID: "9f850214-3252-4e08-9f9d-dedca82385a6") : secret "maas-api-e2e-derive-78079c65-serving-cert" not found Jul 10 21:29:34.211681 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:34.211654 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f850214-3252-4e08-9f9d-dedca82385a6-openshift-service-ca\") pod \"maas-api-e2e-derive-78079c65-75bc99fc76-42jvk\" (UID: \"9f850214-3252-4e08-9f9d-dedca82385a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:34.221723 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:34.221696 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2txd\" (UniqueName: \"kubernetes.io/projected/9f850214-3252-4e08-9f9d-dedca82385a6-kube-api-access-l2txd\") pod \"maas-api-e2e-derive-78079c65-75bc99fc76-42jvk\" (UID: \"9f850214-3252-4e08-9f9d-dedca82385a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:34.714967 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:34.714932 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f850214-3252-4e08-9f9d-dedca82385a6-maas-api-tls\") pod \"maas-api-e2e-derive-78079c65-75bc99fc76-42jvk\" (UID: \"9f850214-3252-4e08-9f9d-dedca82385a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:34.717495 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:34.717464 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f850214-3252-4e08-9f9d-dedca82385a6-maas-api-tls\") pod \"maas-api-e2e-derive-78079c65-75bc99fc76-42jvk\" (UID: \"9f850214-3252-4e08-9f9d-dedca82385a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:34.869124 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:34.869086 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:35.003312 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:35.003146 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk"] Jul 10 21:29:35.006169 ip-10-0-131-227 kubenswrapper[2566]: W0710 21:29:35.006142 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f850214_3252_4e08_9f9d_dedca82385a6.slice/crio-648525b93d27bfa1e70d9a0d943cae37b136c348a1a97e6ea025f92f4369bc6c WatchSource:0}: Error finding container 648525b93d27bfa1e70d9a0d943cae37b136c348a1a97e6ea025f92f4369bc6c: Status 404 returned error can't find the container with id 648525b93d27bfa1e70d9a0d943cae37b136c348a1a97e6ea025f92f4369bc6c Jul 10 21:29:35.636484 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:35.636449 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" event={"ID":"9f850214-3252-4e08-9f9d-dedca82385a6","Type":"ContainerStarted","Data":"7048b6bc69ad45a296ccdb671df3aa8569bb04220c4b1efb324929170e5ed002"} Jul 10 21:29:35.636651 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:35.636500 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" event={"ID":"9f850214-3252-4e08-9f9d-dedca82385a6","Type":"ContainerStarted","Data":"648525b93d27bfa1e70d9a0d943cae37b136c348a1a97e6ea025f92f4369bc6c"} Jul 10 21:29:35.636651 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:35.636548 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:35.653832 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:35.653779 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" podStartSLOduration=2.182897086 podStartE2EDuration="2.653758152s" totalImagesPullingTime="470.861066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:29:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:29:35.652241554 +0000 UTC m=+1909.235937156" watchObservedRunningTime="2026-07-10 21:29:35.653758152 +0000 UTC m=+1909.237453756" Jul 10 21:29:38.181784 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.181753 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk"] Jul 10 21:29:38.182314 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.182040 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" podUID="9f850214-3252-4e08-9f9d-dedca82385a6" containerName="maas-api" containerID="cri-o://7048b6bc69ad45a296ccdb671df3aa8569bb04220c4b1efb324929170e5ed002" gracePeriod=30 Jul 10 21:29:38.439732 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.439671 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:38.550593 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.550561 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2txd\" (UniqueName: \"kubernetes.io/projected/9f850214-3252-4e08-9f9d-dedca82385a6-kube-api-access-l2txd\") pod \"9f850214-3252-4e08-9f9d-dedca82385a6\" (UID: \"9f850214-3252-4e08-9f9d-dedca82385a6\") " Jul 10 21:29:38.550757 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.550642 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f850214-3252-4e08-9f9d-dedca82385a6-openshift-service-ca\") pod \"9f850214-3252-4e08-9f9d-dedca82385a6\" (UID: \"9f850214-3252-4e08-9f9d-dedca82385a6\") " Jul 10 21:29:38.550757 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.550683 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f850214-3252-4e08-9f9d-dedca82385a6-maas-api-tls\") pod \"9f850214-3252-4e08-9f9d-dedca82385a6\" (UID: \"9f850214-3252-4e08-9f9d-dedca82385a6\") " Jul 10 21:29:38.550986 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.550961 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f850214-3252-4e08-9f9d-dedca82385a6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9f850214-3252-4e08-9f9d-dedca82385a6" (UID: "9f850214-3252-4e08-9f9d-dedca82385a6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:29:38.552776 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.552743 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f850214-3252-4e08-9f9d-dedca82385a6-kube-api-access-l2txd" (OuterVolumeSpecName: "kube-api-access-l2txd") pod "9f850214-3252-4e08-9f9d-dedca82385a6" (UID: "9f850214-3252-4e08-9f9d-dedca82385a6"). InnerVolumeSpecName "kube-api-access-l2txd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:29:38.552776 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.552739 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f850214-3252-4e08-9f9d-dedca82385a6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9f850214-3252-4e08-9f9d-dedca82385a6" (UID: "9f850214-3252-4e08-9f9d-dedca82385a6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 21:29:38.651664 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.651634 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f850214-3252-4e08-9f9d-dedca82385a6-maas-api-tls\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:29:38.651664 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.651665 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2txd\" (UniqueName: \"kubernetes.io/projected/9f850214-3252-4e08-9f9d-dedca82385a6-kube-api-access-l2txd\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:29:38.651869 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.651677 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f850214-3252-4e08-9f9d-dedca82385a6-openshift-service-ca\") on node \"ip-10-0-131-227.ec2.internal\" DevicePath \"\"" Jul 10 21:29:38.652517 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.652496 2566 generic.go:356] "Generic (PLEG): container finished" podID="9f850214-3252-4e08-9f9d-dedca82385a6" containerID="7048b6bc69ad45a296ccdb671df3aa8569bb04220c4b1efb324929170e5ed002" exitCode=0 Jul 10 21:29:38.652577 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.652543 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" Jul 10 21:29:38.652623 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.652574 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" event={"ID":"9f850214-3252-4e08-9f9d-dedca82385a6","Type":"ContainerDied","Data":"7048b6bc69ad45a296ccdb671df3aa8569bb04220c4b1efb324929170e5ed002"} Jul 10 21:29:38.652623 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.652613 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk" event={"ID":"9f850214-3252-4e08-9f9d-dedca82385a6","Type":"ContainerDied","Data":"648525b93d27bfa1e70d9a0d943cae37b136c348a1a97e6ea025f92f4369bc6c"} Jul 10 21:29:38.652698 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.652634 2566 scope.go:117] "RemoveContainer" containerID="7048b6bc69ad45a296ccdb671df3aa8569bb04220c4b1efb324929170e5ed002" Jul 10 21:29:38.662341 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.662319 2566 scope.go:117] "RemoveContainer" containerID="7048b6bc69ad45a296ccdb671df3aa8569bb04220c4b1efb324929170e5ed002" Jul 10 21:29:38.662614 ip-10-0-131-227 kubenswrapper[2566]: E0710 21:29:38.662595 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7048b6bc69ad45a296ccdb671df3aa8569bb04220c4b1efb324929170e5ed002\": container with ID starting with 7048b6bc69ad45a296ccdb671df3aa8569bb04220c4b1efb324929170e5ed002 not found: ID does not exist" containerID="7048b6bc69ad45a296ccdb671df3aa8569bb04220c4b1efb324929170e5ed002" Jul 10 21:29:38.662669 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.662623 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7048b6bc69ad45a296ccdb671df3aa8569bb04220c4b1efb324929170e5ed002"} err="failed to get container status \"7048b6bc69ad45a296ccdb671df3aa8569bb04220c4b1efb324929170e5ed002\": rpc error: code = NotFound desc = could not find container \"7048b6bc69ad45a296ccdb671df3aa8569bb04220c4b1efb324929170e5ed002\": container with ID starting with 7048b6bc69ad45a296ccdb671df3aa8569bb04220c4b1efb324929170e5ed002 not found: ID does not exist" Jul 10 21:29:38.674707 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.674684 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk"] Jul 10 21:29:38.678189 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:38.678169 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-78079c65-75bc99fc76-42jvk"] Jul 10 21:29:39.068356 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:29:39.068324 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f850214-3252-4e08-9f9d-dedca82385a6" path="/var/lib/kubelet/pods/9f850214-3252-4e08-9f9d-dedca82385a6/volumes" Jul 10 21:42:34.952050 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:34.952020 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d79d57689-gx8qv_55911c4c-7f7c-46f7-bbec-9dc59a665b49/authorino/0.log" Jul 10 21:42:39.612345 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:39.612259 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7plcl_8ac19b08-450e-40f9-85b0-48e5a3e39174/manager/0.log" Jul 10 21:42:39.731593 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:39.731565 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-22bpn_50226488-7e48-4dc6-b874-0f373bd4200f/manager/0.log" Jul 10 21:42:39.847689 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:39.847656 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5db9df9f4b-zmzfd_41b86611-0dcc-48e9-8c31-4419ea71b58c/manager/0.log" Jul 10 21:42:39.968614 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:39.968587 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6qmtk_c6cf8967-278c-40be-bd75-e0af1e751c6b/server/0.log" Jul 10 21:42:40.082075 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:40.082040 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j5pjd_81dce030-62c0-4144-8b18-9e8642722fbc/manager/0.log" Jul 10 21:42:40.334102 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:40.334016 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58b8cc5f97-k8rt8_4ffb42af-fb8a-4ea8-920e-3a4ce850e983/copy-manifests/0.log" Jul 10 21:42:40.348249 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:40.348218 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58b8cc5f97-k8rt8_4ffb42af-fb8a-4ea8-920e-3a4ce850e983/manager/0.log" Jul 10 21:42:40.815963 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:40.815935 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-65ccc6b695-kjmw8_73e43358-72b1-4db4-9d50-11a17441d1da/maas-api/0.log" Jul 10 21:42:40.937098 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:40.937069 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-6hx8j_6c99a523-0aaa-4d8f-a829-e679b2f1941e/postgres/0.log" Jul 10 21:42:42.215522 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:42.215493 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d79d57689-gx8qv_55911c4c-7f7c-46f7-bbec-9dc59a665b49/authorino/0.log" Jul 10 21:42:42.340756 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:42.340720 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-g78jw_3d49ffc9-3b56-41eb-b6ad-9ef2d49f7d75/manager/0.log" Jul 10 21:42:42.454881 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:42.454850 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-z5fp6_0eef96a5-6001-44b5-b5a8-d38f66be3f42/manager/0.log" Jul 10 21:42:42.732695 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:42.732662 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-lbhcl_714d6812-783b-4e91-a033-04bc634d5062/registry-server/0.log" Jul 10 21:42:42.864180 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:42.864135 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zpf7f_81314945-8ce0-4c51-aeb7-4eac6db2b31b/manager/0.log" Jul 10 21:42:43.135206 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:43.135112 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-j7lzn_32440622-a23a-4e8a-9cca-666f88134ee6/manager/0.log" Jul 10 21:42:43.699001 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:43.698971 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cc6fcfbb4-4kzrx_b614d91b-dc61-4bb6-954f-0307f7da3509/kube-auth-proxy/0.log" Jul 10 21:42:44.043935 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:44.043900 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-l8l59_ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf/payload-pre-processing/0.log" Jul 10 21:42:44.156091 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:44.156058 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-rttt6_2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd/payload-processing/0.log" Jul 10 21:42:44.730957 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:44.730926 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-rrz8t_16b3518d-d35b-4e26-86e6-e312bfaafd92/main/0.log" Jul 10 21:42:45.183561 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:45.183476 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-7wch8_0ceb18bc-3c9a-481d-acce-2219d0ed3c12/main/0.log" Jul 10 21:42:52.338660 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:52.338627 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9788m_94f8ad00-553c-47e0-8453-cdbe5b2a2b39/global-pull-secret-syncer/0.log" Jul 10 21:42:52.545034 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:52.545007 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nmqk9_336c95f4-fe3d-4af2-b2bf-395b7cae9633/konnectivity-agent/0.log" Jul 10 21:42:52.566553 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:52.566529 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-227.ec2.internal_1cad2d201741314c351e55ceddd77eeb/haproxy/0.log" Jul 10 21:42:57.505873 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:57.505845 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d79d57689-gx8qv_55911c4c-7f7c-46f7-bbec-9dc59a665b49/authorino/0.log" Jul 10 21:42:57.547452 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:57.547423 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-g78jw_3d49ffc9-3b56-41eb-b6ad-9ef2d49f7d75/manager/0.log" Jul 10 21:42:57.579290 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:57.579265 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-z5fp6_0eef96a5-6001-44b5-b5a8-d38f66be3f42/manager/0.log" Jul 10 21:42:57.675114 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:57.675089 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-lbhcl_714d6812-783b-4e91-a033-04bc634d5062/registry-server/0.log" Jul 10 21:42:57.810812 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:57.810733 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zpf7f_81314945-8ce0-4c51-aeb7-4eac6db2b31b/manager/0.log" Jul 10 21:42:57.934454 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:42:57.934413 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-j7lzn_32440622-a23a-4e8a-9cca-666f88134ee6/manager/0.log" Jul 10 21:43:00.108472 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:00.108441 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwwvt_8d5896fa-3c3f-43d8-963f-5705467f838d/node-exporter/0.log" Jul 10 21:43:00.130893 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:00.130870 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwwvt_8d5896fa-3c3f-43d8-963f-5705467f838d/kube-rbac-proxy/0.log" Jul 10 21:43:00.153821 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:00.153802 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pwwvt_8d5896fa-3c3f-43d8-963f-5705467f838d/init-textfile/0.log" Jul 10 21:43:02.043799 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.043765 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg"] Jul 10 21:43:02.044285 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.044272 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f850214-3252-4e08-9f9d-dedca82385a6" containerName="maas-api" Jul 10 21:43:02.044353 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.044292 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f850214-3252-4e08-9f9d-dedca82385a6" containerName="maas-api" Jul 10 21:43:02.044437 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.044425 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f850214-3252-4e08-9f9d-dedca82385a6" containerName="maas-api" Jul 10 21:43:02.047709 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.047689 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.050214 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.050195 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mxzgk\"/\"default-dockercfg-766q9\"" Jul 10 21:43:02.050407 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.050371 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mxzgk\"/\"openshift-service-ca.crt\"" Jul 10 21:43:02.051046 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.051030 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mxzgk\"/\"kube-root-ca.crt\"" Jul 10 21:43:02.058721 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.058699 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg"] Jul 10 21:43:02.165374 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.165345 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8aae27e0-e7b1-429b-a431-4596a0936f91-proc\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.165555 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.165489 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8aae27e0-e7b1-429b-a431-4596a0936f91-sys\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.165555 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.165523 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9n9z\" (UniqueName: \"kubernetes.io/projected/8aae27e0-e7b1-429b-a431-4596a0936f91-kube-api-access-l9n9z\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.165636 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.165565 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8aae27e0-e7b1-429b-a431-4596a0936f91-lib-modules\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.165718 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.165695 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8aae27e0-e7b1-429b-a431-4596a0936f91-podres\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.266626 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.266594 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8aae27e0-e7b1-429b-a431-4596a0936f91-proc\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.266813 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.266666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8aae27e0-e7b1-429b-a431-4596a0936f91-sys\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.266813 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.266689 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9n9z\" (UniqueName: \"kubernetes.io/projected/8aae27e0-e7b1-429b-a431-4596a0936f91-kube-api-access-l9n9z\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.266813 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.266720 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8aae27e0-e7b1-429b-a431-4596a0936f91-lib-modules\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.266813 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.266724 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8aae27e0-e7b1-429b-a431-4596a0936f91-proc\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.266813 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.266779 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8aae27e0-e7b1-429b-a431-4596a0936f91-sys\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.266813 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.266789 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8aae27e0-e7b1-429b-a431-4596a0936f91-podres\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.267048 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.266866 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8aae27e0-e7b1-429b-a431-4596a0936f91-lib-modules\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.267048 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.266870 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8aae27e0-e7b1-429b-a431-4596a0936f91-podres\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.275675 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.275654 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9n9z\" (UniqueName: \"kubernetes.io/projected/8aae27e0-e7b1-429b-a431-4596a0936f91-kube-api-access-l9n9z\") pod \"perf-node-gather-daemonset-wfzhg\" (UID: \"8aae27e0-e7b1-429b-a431-4596a0936f91\") " pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.359024 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.358936 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:02.514338 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:02.514298 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg"] Jul 10 21:43:03.334498 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:03.334457 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" event={"ID":"8aae27e0-e7b1-429b-a431-4596a0936f91","Type":"ContainerStarted","Data":"5ed0934d793c9af3358a00e7249431e0f3eb6d1a011ceabc4c5e1f363c600bcb"} Jul 10 21:43:03.334923 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:03.334515 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" event={"ID":"8aae27e0-e7b1-429b-a431-4596a0936f91","Type":"ContainerStarted","Data":"20f45446a85e890509e31e0c92a5850654a7a55f8d944a539af28c1f3f6dee35"} Jul 10 21:43:03.334923 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:03.334565 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:03.351442 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:03.351372 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" podStartSLOduration=1.3513556389999999 podStartE2EDuration="1.351355639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 21:43:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:43:03.350005074 +0000 UTC m=+2716.933700688" watchObservedRunningTime="2026-07-10 21:43:03.351355639 +0000 UTC m=+2716.935051241" Jul 10 21:43:05.007702 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:05.007671 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fd78g_cd7ffba8-211e-43b8-bfe6-92696ef7035b/dns/0.log" Jul 10 21:43:05.031044 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:05.031019 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fd78g_cd7ffba8-211e-43b8-bfe6-92696ef7035b/kube-rbac-proxy/0.log" Jul 10 21:43:05.155644 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:05.155614 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-62r4h_3ba53ae5-7a20-4763-a164-0924edc26ba2/dns-node-resolver/0.log" Jul 10 21:43:05.854121 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:05.854093 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rnz7t_e973bb49-d732-4f0d-b518-7f25d5ebd9ff/node-ca/0.log" Jul 10 21:43:06.898513 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:06.898483 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cc6fcfbb4-4kzrx_b614d91b-dc61-4bb6-954f-0307f7da3509/kube-auth-proxy/0.log" Jul 10 21:43:07.021855 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:07.021823 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-l8l59_ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf/payload-pre-processing/0.log" Jul 10 21:43:07.053024 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:07.052982 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-rttt6_2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd/payload-processing/0.log" Jul 10 21:43:07.732886 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:07.732860 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-84gxc_69b2e978-68f0-4f4d-acf2-1cc9ef90cd28/serve-healthcheck-canary/0.log" Jul 10 21:43:08.389924 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:08.389896 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mpbf9_ff0579cc-b6f0-4cea-9840-0850d7fe6c31/kube-rbac-proxy/0.log" Jul 10 21:43:08.414341 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:08.414318 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mpbf9_ff0579cc-b6f0-4cea-9840-0850d7fe6c31/exporter/0.log" Jul 10 21:43:08.438048 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:08.438025 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mpbf9_ff0579cc-b6f0-4cea-9840-0850d7fe6c31/extractor/0.log" Jul 10 21:43:09.348801 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:09.348769 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mxzgk/perf-node-gather-daemonset-wfzhg" Jul 10 21:43:10.754463 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:10.754437 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7plcl_8ac19b08-450e-40f9-85b0-48e5a3e39174/manager/0.log" Jul 10 21:43:10.838414 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:10.838366 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-22bpn_50226488-7e48-4dc6-b874-0f373bd4200f/manager/0.log" Jul 10 21:43:10.916662 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:10.916623 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5db9df9f4b-zmzfd_41b86611-0dcc-48e9-8c31-4419ea71b58c/manager/0.log" Jul 10 21:43:10.942849 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:10.942768 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6qmtk_c6cf8967-278c-40be-bd75-e0af1e751c6b/server/0.log" Jul 10 21:43:10.998688 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:10.998664 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j5pjd_81dce030-62c0-4144-8b18-9e8642722fbc/manager/0.log" Jul 10 21:43:11.123248 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:11.123213 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58b8cc5f97-k8rt8_4ffb42af-fb8a-4ea8-920e-3a4ce850e983/manager/0.log" Jul 10 21:43:11.147906 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:11.147875 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58b8cc5f97-k8rt8_4ffb42af-fb8a-4ea8-920e-3a4ce850e983/copy-manifests/0.log" Jul 10 21:43:12.472903 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:12.472875 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cc6fcfbb4-4kzrx_b614d91b-dc61-4bb6-954f-0307f7da3509/kube-auth-proxy/0.log" Jul 10 21:43:12.559652 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:12.559622 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-l8l59_ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf/payload-pre-processing/0.log" Jul 10 21:43:12.580671 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:12.580643 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-rttt6_2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd/payload-processing/0.log" Jul 10 21:43:13.868654 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:13.868571 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cc6fcfbb4-4kzrx_b614d91b-dc61-4bb6-954f-0307f7da3509/kube-auth-proxy/0.log" Jul 10 21:43:13.961763 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:13.961731 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-l8l59_ef8fb89d-4eb9-44cb-86b9-f3e45863fbaf/payload-pre-processing/0.log" Jul 10 21:43:13.982027 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:13.981999 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-rttt6_2dcc9082-e9cd-43d3-8e9c-fa64ae45a9cd/payload-processing/0.log" Jul 10 21:43:19.380737 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:19.380710 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6kpsn_3fc26a94-4249-49e4-af9c-f4deea65bc50/kube-multus/0.log" Jul 10 21:43:19.590008 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:19.589981 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kv7p8_caad5f13-3ab6-4db2-bfbe-c5e9781d5f76/kube-multus-additional-cni-plugins/0.log" Jul 10 21:43:19.614837 ip-10-0-131-227 kubenswrapper[2566]: I0710 21:43:19.614802 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kv7p8_caad5f13-3ab6-4db2-bfbe-c5e9781d5f76/egress-router-binary-copy/0.log"