Jul 23 15:12:26.266589 ip-10-0-129-63 systemd[1]: Starting Kubernetes Kubelet... Jul 23 15:12:27.983218 ip-10-0-129-63 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 15:12:27.983218 ip-10-0-129-63 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 15:12:27.983218 ip-10-0-129-63 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 15:12:27.983218 ip-10-0-129-63 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 15:12:27.983218 ip-10-0-129-63 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 15:12:27.983218 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.905710 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 15:12:27.983218 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915096 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 15:12:27.983218 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915110 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 15:12:27.983218 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915114 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 15:12:27.983218 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915118 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 15:12:27.983218 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915121 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 15:12:27.983218 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915124 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 15:12:27.983218 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915127 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 15:12:27.983218 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915130 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915133 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915135 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915138 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915141 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915144 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915146 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915149 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915152 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915154 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915157 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915160 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915165 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915171 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915175 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915179 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915183 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915186 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915190 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 15:12:27.989558 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915194 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915197 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915200 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915203 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915206 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915209 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915212 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915216 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915220 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915223 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915225 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915228 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915231 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915234 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915240 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915242 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915245 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915248 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915251 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915254 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 15:12:27.990401 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915257 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915260 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915263 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915265 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915268 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915271 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915274 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915277 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915280 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915283 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915286 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915289 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915293 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915297 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915302 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915305 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915308 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915312 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915315 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915318 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 15:12:27.991227 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915321 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915324 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915327 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915330 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915332 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915336 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915338 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915341 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915344 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915347 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915350 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915353 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915355 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915358 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915360 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915363 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915366 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915384 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915388 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 15:12:27.997187 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915391 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 15:12:27.995293 ip-10-0-129-63 systemd[1]: Started Kubernetes Kubelet. Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915394 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915397 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915400 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915403 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915407 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915410 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915413 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915415 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915418 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915421 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915424 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915427 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.915430 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916219 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916226 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916229 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916233 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916235 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 15:12:27.998352 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916239 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916242 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916246 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916249 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916252 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916255 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916257 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916260 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916263 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916265 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916270 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916273 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916278 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916281 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916284 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916286 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916289 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916292 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916295 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 15:12:27.999860 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916298 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916301 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916303 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916306 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916309 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916312 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916315 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916318 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916322 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916326 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916329 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916332 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916335 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916339 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916343 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916346 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916349 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916352 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916355 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916357 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 15:12:28.003707 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916360 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916364 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916366 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916390 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916393 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916396 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916399 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916402 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916405 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916407 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916411 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916414 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916416 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916419 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916422 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916424 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916427 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916430 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916433 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916435 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 15:12:28.005192 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916438 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916441 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916445 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916448 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916450 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916453 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916456 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916460 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916462 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916465 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916468 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916471 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916474 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916477 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916480 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916483 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916486 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916489 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916491 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916494 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 15:12:28.008993 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916497 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916499 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916502 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916506 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916508 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916511 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916515 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916517 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916520 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916523 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916526 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916530 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916533 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916537 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916540 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916953 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916958 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916962 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916965 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 15:12:28.009934 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916969 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916972 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916977 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916980 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916983 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916985 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916990 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916992 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916995 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.916998 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917001 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917004 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917007 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917010 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917013 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917016 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917019 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917022 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917025 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917028 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 15:12:28.121676 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917030 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917033 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917036 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917039 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917041 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917044 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917047 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917050 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917053 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917055 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917058 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917061 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917063 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917067 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917071 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917075 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917079 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917082 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917087 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 15:12:28.122497 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917090 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917093 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917096 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917100 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917102 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917105 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917109 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917112 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917115 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917118 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917121 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917125 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917129 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917132 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917134 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917137 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917140 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917143 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917145 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 15:12:28.234019 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917148 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917150 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917153 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917156 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917158 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917161 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917164 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917166 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917171 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917174 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917177 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917179 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917184 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917186 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917189 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917192 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917195 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917199 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917201 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917204 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 15:12:28.235270 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917207 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917210 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917212 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917215 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917218 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917221 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917223 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917226 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917229 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917232 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917235 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917237 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917240 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917242 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917245 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917248 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.917250 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917341 2570 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917349 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917356 2570 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917364 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 15:12:28.236131 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917384 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917389 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917394 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917400 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917404 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917407 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917411 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917415 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917418 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917421 2570 flags.go:64] FLAG: --cgroup-root="" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917425 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917428 2570 flags.go:64] FLAG: --client-ca-file="" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917431 2570 flags.go:64] FLAG: --cloud-provider="external" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917434 2570 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917439 2570 flags.go:64] FLAG: --cluster-domain="" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917442 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917445 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917449 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917452 2570 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917457 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917461 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917464 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917468 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917471 2570 flags.go:64] FLAG: --contention-profiling="false" Jul 23 15:12:28.237804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917474 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917477 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917480 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917484 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917488 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917491 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917494 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917499 2570 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917502 2570 flags.go:64] FLAG: --enable-server="true" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917506 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917511 2570 flags.go:64] FLAG: --event-burst="100" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917516 2570 flags.go:64] FLAG: --event-qps="50" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917520 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917523 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917526 2570 flags.go:64] FLAG: --eviction-hard="" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917531 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917534 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917537 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917540 2570 flags.go:64] FLAG: --eviction-soft="" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917544 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917547 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917550 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917553 2570 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917556 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917559 2570 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 15:12:28.238704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917562 2570 flags.go:64] FLAG: --feature-gates="" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917566 2570 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917570 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917573 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917576 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917580 2570 flags.go:64] FLAG: --healthz-port="10248" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917583 2570 flags.go:64] FLAG: --help="false" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917586 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917589 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917592 2570 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917595 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917599 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917603 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917606 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917610 2570 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917613 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917616 2570 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917620 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917625 2570 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917628 2570 flags.go:64] FLAG: --kube-reserved="" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917631 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917635 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917638 2570 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917641 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 15:12:28.239978 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917644 2570 flags.go:64] FLAG: --lock-file="" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917647 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917651 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917654 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917660 2570 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917663 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917666 2570 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917669 2570 flags.go:64] FLAG: --logging-format="text" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917672 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917676 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917679 2570 flags.go:64] FLAG: --manifest-url="" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917682 2570 flags.go:64] FLAG: --manifest-url-header="" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917687 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917690 2570 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917695 2570 flags.go:64] FLAG: --max-pods="110" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917698 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917701 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917704 2570 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917707 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917710 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917714 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917717 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917727 2570 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917731 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917734 2570 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 15:12:28.240819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917737 2570 flags.go:64] FLAG: --pod-cidr="" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917743 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917749 2570 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917752 2570 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917755 2570 flags.go:64] FLAG: --pods-per-core="0" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917759 2570 flags.go:64] FLAG: --port="10250" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917762 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917765 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02c9e21cc019cc870" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917769 2570 flags.go:64] FLAG: --qos-reserved="" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917772 2570 flags.go:64] FLAG: --read-only-port="10255" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917775 2570 flags.go:64] FLAG: --register-node="true" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917779 2570 flags.go:64] FLAG: --register-with-taints="" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917783 2570 flags.go:64] FLAG: --registry-burst="10" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917786 2570 flags.go:64] FLAG: --registry-qps="5" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917789 2570 flags.go:64] FLAG: --reserved-cpus="" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917792 2570 flags.go:64] FLAG: --reserved-memory="" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917796 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917800 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917803 2570 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917806 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917810 2570 flags.go:64] FLAG: --runonce="false" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917813 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917816 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917819 2570 flags.go:64] FLAG: --seccomp-default="false" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917822 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 15:12:28.242090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917825 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917829 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917832 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917835 2570 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917840 2570 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917843 2570 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917846 2570 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917849 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917852 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917857 2570 flags.go:64] FLAG: --system-cgroups="" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917860 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917866 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917870 2570 flags.go:64] FLAG: --tls-cert-file="" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917873 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917878 2570 flags.go:64] FLAG: --tls-min-version="" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917881 2570 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917885 2570 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917888 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917891 2570 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917894 2570 flags.go:64] FLAG: --v="2" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917899 2570 flags.go:64] FLAG: --version="false" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917903 2570 flags.go:64] FLAG: --vmodule="" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917907 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.917911 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918036 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 15:12:28.244294 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918040 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918043 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918046 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918049 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918052 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918057 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918060 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918063 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918067 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918069 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918072 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918076 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918080 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918083 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918086 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918089 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918093 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918096 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918100 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918102 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 15:12:28.245033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918106 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918109 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918112 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918115 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918117 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918120 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918123 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918126 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918128 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918131 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918134 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918136 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918140 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918142 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918147 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918151 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918153 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918156 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918159 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 15:12:28.245575 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918162 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918164 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918167 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918170 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918176 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918179 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918182 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918185 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918187 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918192 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918195 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918199 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918201 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918204 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918207 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918210 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918213 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918216 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918219 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 15:12:28.246108 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918222 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918224 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918227 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918230 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918233 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918236 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918238 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918241 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918244 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918247 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918250 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918252 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918255 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918258 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918261 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918264 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918268 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918272 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918275 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 15:12:28.246617 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918277 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918280 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918283 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918288 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918290 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918293 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918296 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918299 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918302 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918305 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918307 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918310 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918313 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918315 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918318 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918321 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918324 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918326 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918329 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918332 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 15:12:28.247088 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.918335 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 15:12:28.247626 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.919182 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 15:12:28.247626 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.927018 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 15:12:28.247626 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.927034 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 15:12:28.247626 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.927850 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 15:12:28.247626 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.927859 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 15:12:28.247626 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927922 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 15:12:28.247626 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927928 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 15:12:28.247626 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927932 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 23 15:12:28.247626 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927937 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 15:12:28.247626 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927942 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 15:12:28.247626 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927945 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 15:12:28.247626 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927949 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 15:12:28.247626 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927952 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 15:12:28.247626 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927955 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927959 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927962 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927965 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927969 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927972 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927975 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927978 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927981 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927984 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927986 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927989 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927992 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927995 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.927998 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928001 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928004 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928007 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928010 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928013 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 15:12:28.248008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928016 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928020 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928023 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928025 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928028 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928031 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928033 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928036 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928039 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928043 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928046 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928049 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928052 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928055 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928057 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928060 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928063 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928066 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928069 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 15:12:28.248538 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928071 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928074 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928077 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928080 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928082 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928085 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928089 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928093 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928096 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928099 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928101 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928104 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928107 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928110 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928113 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928116 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928119 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928122 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928125 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 15:12:28.249027 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928128 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928131 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928134 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928136 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928139 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928142 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928144 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928147 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928150 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928153 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928155 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928158 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928161 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928164 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928166 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928169 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928172 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928175 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928177 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928180 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 15:12:28.249526 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928183 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928186 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928188 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928191 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928194 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928196 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928200 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928206 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928208 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928211 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928214 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928217 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928219 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.928225 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928359 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 15:12:28.250034 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928365 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928369 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928373 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928378 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928382 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928386 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928388 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928392 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928395 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928398 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928403 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928406 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928409 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928412 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928415 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928418 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928423 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928425 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928428 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928431 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 15:12:28.250441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928434 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928437 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928439 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928442 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928446 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928449 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928451 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928454 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928457 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928460 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928462 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928465 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928469 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928473 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928476 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928479 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928484 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928487 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928491 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 15:12:28.250953 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928494 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928497 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928500 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928503 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928506 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928509 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928512 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928514 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928517 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928521 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928524 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928526 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928529 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928532 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928535 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928537 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928540 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928544 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928547 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928550 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 15:12:28.251452 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928552 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928555 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928558 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928561 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928563 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928566 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928569 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928573 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928577 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928580 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928583 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928586 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928589 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928592 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928595 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928598 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928601 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928604 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928607 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 15:12:28.252008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928610 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928613 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928617 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928620 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928623 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928626 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928629 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928632 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928635 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928637 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928641 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928644 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928647 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928649 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928652 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928655 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928658 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928660 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928663 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 15:12:28.252529 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:27.928666 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 15:12:28.253010 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.928671 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 15:12:28.253010 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.928782 2570 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 15:12:28.253010 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.930928 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 15:12:28.253010 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.932503 2570 server.go:1019] "Starting client certificate rotation" Jul 23 15:12:28.253010 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.932611 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 15:12:28.253010 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.932645 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 15:12:28.253010 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.935346 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 15:12:28.253010 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.940397 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 15:12:28.253010 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.956567 2570 log.go:25] "Validated CRI v1 runtime API" Jul 23 15:12:28.253010 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.958787 2570 log.go:25] "Validated CRI v1 image API" Jul 23 15:12:28.253010 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.959716 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 15:12:28.253010 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.966288 2570 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d20074a2-047b-41ab-ae6f-697e97d0d0c4:/dev/nvme0n1p3 dfab98e3-e09b-4068-95c4-50c08c4daee4:/dev/nvme0n1p4] Jul 23 15:12:28.253344 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.966311 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 15:12:28.253344 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.967162 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.971643 2570 manager.go:216] Machine: {Timestamp:2026-07-23 15:12:27.971111553 +0000 UTC m=+1.587158811 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101010 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22f190a3bb2a542fcf6a3c609d302b SystemUUID:ec22f190-a3bb-2a54-2fcf-6a3c609d302b BootID:81647eec-e02c-4d22-b1a7-3bd7e58221c1 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a6:18:6b:b2:ad Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a6:18:6b:b2:ad Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:f6:83:fc:87:9e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.971752 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.971879 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.975317 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.975346 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-63.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.976804 2570 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.976816 2570 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.976830 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.977972 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.979386 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.983045 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.983178 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.983209 2570 kubelet.go:486] "Attempting to sync node with API server" Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.983221 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.983236 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.983246 2570 kubelet.go:398] "Adding apiserver pod source" Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.983255 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 15:12:28.253426 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.983748 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.983775 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.986819 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.988782 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.988813 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.989975 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.989989 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.989996 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.990002 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.990012 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.990018 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.990027 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.990032 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.990040 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.990046 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.990056 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.990062 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.990800 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.990810 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.994359 2570 server.go:1268] "Started kubelet" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.994647 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.994693 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.999047 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:27.999647 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.001476 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-63.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.001994 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-63.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.002155 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.004327 2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.005555 2570 server.go:310] "Adding debug handlers to kubelet server" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.009286 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-63.ec2.internal.18c4f3ec2a5ff5ff default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-63.ec2.internal,UID:ip-10-0-129-63.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-63.ec2.internal,},FirstTimestamp:2026-07-23 15:12:27.994322431 +0000 UTC m=+1.610369689,LastTimestamp:2026-07-23 15:12:27.994322431 +0000 UTC m=+1.610369689,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-63.ec2.internal,}" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.010434 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.010458 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.010549 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-63.ec2.internal\" not found" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.011243 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.011268 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.011280 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.011391 2570 reconstruct.go:97] "Volume reconstruction finished" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.011403 2570 reconciler.go:29] "Reconciler: start to sync state" Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.011559 2570 factory.go:55] Registering systemd factory Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.011603 2570 factory.go:223] Registration of the systemd container factory successfully Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.011846 2570 factory.go:153] Registering CRI-O factory Jul 23 15:12:28.253930 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.011860 2570 factory.go:223] Registration of the crio container factory successfully Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.011928 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.011953 2570 factory.go:103] Registering Raw factory Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.011969 2570 manager.go:1196] Started watching for new ooms in manager Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.012385 2570 manager.go:349] Starting recovery of all containers Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.023113 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-63.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.023330 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.023800 2570 manager.go:354] Recovery completed Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.029948 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.031983 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.032017 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.032033 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.032615 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.032626 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.032635 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.032642 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.033379 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.033406 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.033418 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.034023 2570 policy_none.go:49] "None policy: Start" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.034039 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.034053 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.035087 2570 policy_none.go:47] "Start" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.036694 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f7r26" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.042278 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-63.ec2.internal.18c4f3ec2c9ed96c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-63.ec2.internal,UID:ip-10-0-129-63.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-63.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-63.ec2.internal,},FirstTimestamp:2026-07-23 15:12:28.031998316 +0000 UTC m=+1.648045583,LastTimestamp:2026-07-23 15:12:28.031998316 +0000 UTC m=+1.648045583,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-63.ec2.internal,}" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.046280 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f7r26" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.111006 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-63.ec2.internal\" not found" Jul 23 15:12:28.254996 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.211439 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-63.ec2.internal\" not found" Jul 23 15:12:28.300762 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.300728 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-63.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms" Jul 23 15:12:28.300890 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.300743 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-63.ec2.internal.18c4f3ec2c9f421a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-63.ec2.internal,UID:ip-10-0-129-63.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-63.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-63.ec2.internal,},FirstTimestamp:2026-07-23 15:12:28.032025114 +0000 UTC m=+1.648072372,LastTimestamp:2026-07-23 15:12:28.032025114 +0000 UTC m=+1.648072372,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-63.ec2.internal,}" Jul 23 15:12:28.312594 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.312560 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-63.ec2.internal\" not found" Jul 23 15:12:28.349271 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.349003 2570 manager.go:337] "Starting Device Plugin manager" Jul 23 15:12:28.349271 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.349237 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 15:12:28.349271 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.349254 2570 server.go:87] "Starting device plugin registration server" Jul 23 15:12:28.349607 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.349592 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 15:12:28.349668 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.349608 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 15:12:28.349730 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.349707 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 15:12:28.349833 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.349817 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 15:12:28.349833 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.349832 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 15:12:28.350890 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.350871 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 15:12:28.350968 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.350914 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-63.ec2.internal\" not found" Jul 23 15:12:28.430482 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.430446 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 15:12:28.432443 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.431802 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 15:12:28.432443 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.431840 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 15:12:28.432443 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.431862 2570 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 15:12:28.432443 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.431900 2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 15:12:28.437103 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.437082 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 15:12:28.450688 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.450674 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:12:28.451650 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.451629 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:12:28.451730 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.451661 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:12:28.451730 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.451673 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:12:28.451730 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.451699 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.463729 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.463710 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.463781 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.463731 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-63.ec2.internal\": node \"ip-10-0-129-63.ec2.internal\" not found" Jul 23 15:12:28.486874 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.486846 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-63.ec2.internal\" not found" Jul 23 15:12:28.532679 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.532641 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-63.ec2.internal"] Jul 23 15:12:28.532789 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.532721 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:12:28.533804 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.533788 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:12:28.533889 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.533825 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:12:28.533889 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.533842 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:12:28.535119 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.535107 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:12:28.535237 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.535224 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.535276 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.535256 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:12:28.535886 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.535870 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:12:28.535962 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.535891 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:12:28.535962 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.535912 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:12:28.535962 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.535921 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:12:28.535962 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.535893 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:12:28.536109 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.535991 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:12:28.537714 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.537701 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.537776 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.537727 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:12:28.538436 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.538418 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:12:28.538498 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.538456 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:12:28.538498 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.538474 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:12:28.563287 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.563258 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-63.ec2.internal\" not found" node="ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.567125 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.567108 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-63.ec2.internal\" not found" node="ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.587361 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.587332 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-63.ec2.internal\" not found" Jul 23 15:12:28.614830 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.614801 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/67b7466e3c1de219c8d3784cc786446a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal\" (UID: \"67b7466e3c1de219c8d3784cc786446a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.614960 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.614837 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67b7466e3c1de219c8d3784cc786446a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal\" (UID: \"67b7466e3c1de219c8d3784cc786446a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.614960 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.614858 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b932a6350e4894a972718b80947507d2-config\") pod \"kube-apiserver-proxy-ip-10-0-129-63.ec2.internal\" (UID: \"b932a6350e4894a972718b80947507d2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.688013 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.687920 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-63.ec2.internal\" not found" Jul 23 15:12:28.715635 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.715608 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/67b7466e3c1de219c8d3784cc786446a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal\" (UID: \"67b7466e3c1de219c8d3784cc786446a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.715794 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.715642 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67b7466e3c1de219c8d3784cc786446a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal\" (UID: \"67b7466e3c1de219c8d3784cc786446a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.715794 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.715664 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b932a6350e4894a972718b80947507d2-config\") pod \"kube-apiserver-proxy-ip-10-0-129-63.ec2.internal\" (UID: \"b932a6350e4894a972718b80947507d2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.715794 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.715716 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b932a6350e4894a972718b80947507d2-config\") pod \"kube-apiserver-proxy-ip-10-0-129-63.ec2.internal\" (UID: \"b932a6350e4894a972718b80947507d2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.715794 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.715723 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/67b7466e3c1de219c8d3784cc786446a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal\" (UID: \"67b7466e3c1de219c8d3784cc786446a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.715794 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.715734 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67b7466e3c1de219c8d3784cc786446a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal\" (UID: \"67b7466e3c1de219c8d3784cc786446a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.788527 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.788487 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-63.ec2.internal\" not found" Jul 23 15:12:28.869982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.869933 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.869982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.869980 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-63.ec2.internal" Jul 23 15:12:28.889350 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.889325 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-63.ec2.internal\" not found" Jul 23 15:12:28.931835 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.931803 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 15:12:28.931967 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:28.931936 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 15:12:28.990288 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:28.990227 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-63.ec2.internal\" not found" Jul 23 15:12:29.051181 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.051142 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 15:07:28 +0000 UTC" deadline="2028-01-20 05:43:08.258165063 +0000 UTC" Jul 23 15:12:29.051181 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.051173 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13094h30m39.206995176s" Jul 23 15:12:29.091330 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:29.091288 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-63.ec2.internal\" not found" Jul 23 15:12:29.109479 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.109448 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:12:29.110529 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.110514 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:12:29.110613 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.110546 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:12:29.110613 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.110561 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:12:29.187087 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.187053 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 15:12:29.191547 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:29.191529 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-63.ec2.internal\" not found" Jul 23 15:12:29.212005 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.211984 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 15:12:29.311133 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.311064 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal" Jul 23 15:12:29.334791 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.334765 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 15:12:29.335915 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.335898 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-63.ec2.internal" Jul 23 15:12:29.350659 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.350641 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 15:12:29.465592 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:29.465556 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb932a6350e4894a972718b80947507d2.slice/crio-560b559c1f341c2766eceba5736d5c9f5dbdd5356841b731c21f69cb5ffaa79d WatchSource:0}: Error finding container 560b559c1f341c2766eceba5736d5c9f5dbdd5356841b731c21f69cb5ffaa79d: Status 404 returned error can't find the container with id 560b559c1f341c2766eceba5736d5c9f5dbdd5356841b731c21f69cb5ffaa79d Jul 23 15:12:29.469551 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.469535 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 15:12:29.500209 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.500189 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 15:12:29.553312 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:29.553266 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b7466e3c1de219c8d3784cc786446a.slice/crio-e33b8c28a9de75e7a9d4050cefe2ad36955a5cc652775faf54abc49ec3c00ce8 WatchSource:0}: Error finding container e33b8c28a9de75e7a9d4050cefe2ad36955a5cc652775faf54abc49ec3c00ce8: Status 404 returned error can't find the container with id e33b8c28a9de75e7a9d4050cefe2ad36955a5cc652775faf54abc49ec3c00ce8 Jul 23 15:12:29.985000 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.984965 2570 apiserver.go:52] "Watching apiserver" Jul 23 15:12:29.995207 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.995173 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 15:12:29.997197 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.997163 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv","openshift-cluster-node-tuning-operator/tuned-njgnh","openshift-image-registry/node-ca-k57jv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal","openshift-multus/multus-additional-cni-plugins-4bhpc","openshift-multus/multus-rdgrb","openshift-multus/network-metrics-daemon-n5624","openshift-network-diagnostics/network-check-target-ltmrf","kube-system/global-pull-secret-syncer-wl9qs","kube-system/konnectivity-agent-vnw4c","kube-system/kube-apiserver-proxy-ip-10-0-129-63.ec2.internal","openshift-network-operator/iptables-alerter-65k22","openshift-ovn-kubernetes/ovnkube-node-cdjsd"] Jul 23 15:12:29.998840 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:29.998813 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vnw4c" Jul 23 15:12:30.000411 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.000384 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.002385 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.002365 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k57jv" Jul 23 15:12:30.003944 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.003466 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 15:12:30.003944 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.003720 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 15:12:30.004680 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.004514 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.005583 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.005541 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:12:30.005909 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.005890 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 15:12:30.006102 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.006089 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.007466 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.007385 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:30.007466 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.007448 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:30.008643 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.008622 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:30.008743 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.008692 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:30.010215 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.010198 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:30.010320 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.010269 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:30.011621 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.011439 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 15:12:30.014416 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.011810 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.014416 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.012612 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 15:12:30.014416 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.014196 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 15:12:30.014416 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.014324 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 15:12:30.014800 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.014583 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 15:12:30.015497 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.014975 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-65k22" Jul 23 15:12:30.015497 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.015177 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5gbzm\"" Jul 23 15:12:30.015497 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.015190 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.015701 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.015686 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8c4jh\"" Jul 23 15:12:30.015751 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.015701 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 15:12:30.015808 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.015793 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 15:12:30.015855 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.015828 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 15:12:30.016185 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.015950 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 15:12:30.016185 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.016050 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-v6v4z\"" Jul 23 15:12:30.016989 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.016970 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 15:12:30.019215 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.018705 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pt2cw\"" Jul 23 15:12:30.019215 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.018840 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6ch26\"" Jul 23 15:12:30.019215 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.018860 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 15:12:30.019215 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.018983 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bxfq5\"" Jul 23 15:12:30.019215 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.019004 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:12:30.019215 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.019015 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 15:12:30.019215 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.019130 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 15:12:30.019215 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.019142 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 15:12:30.022024 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.021974 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 15:12:30.022162 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.022133 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 15:12:30.022922 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.022890 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 15:12:30.023031 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.022951 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-etc-selinux\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.023031 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.022981 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8028e717-3208-45d4-a49f-117180037b85-cnibin\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.023031 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023018 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.023197 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023045 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-sysctl-conf\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.023197 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023068 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-device-dir\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.023197 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023089 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed2fe790-e71b-4c4f-885f-0a5661328a04-host-slash\") pod \"iptables-alerter-65k22\" (UID: \"ed2fe790-e71b-4c4f-885f-0a5661328a04\") " pod="openshift-network-operator/iptables-alerter-65k22" Jul 23 15:12:30.023197 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023094 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mzbpw\"" Jul 23 15:12:30.023197 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023111 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tns7v\" (UniqueName: \"kubernetes.io/projected/7b189e73-6f40-467e-9bf1-5505c5a7194c-kube-api-access-tns7v\") pod \"network-metrics-daemon-n5624\" (UID: \"7b189e73-6f40-467e-9bf1-5505c5a7194c\") " pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:30.023197 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023138 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-socket-dir\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.023197 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023161 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qj62\" (UniqueName: \"kubernetes.io/projected/8322eaa3-685b-44b5-834f-ae04b6ec6566-kube-api-access-8qj62\") pod \"node-ca-k57jv\" (UID: \"8322eaa3-685b-44b5-834f-ae04b6ec6566\") " pod="openshift-image-registry/node-ca-k57jv" Jul 23 15:12:30.023197 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023185 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-os-release\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.023600 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023207 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-run-k8s-cni-cncf-io\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.023600 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023261 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-multus-conf-dir\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.023600 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023310 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-etc-kubernetes\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.023600 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023338 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kwgj\" (UniqueName: \"kubernetes.io/projected/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-kube-api-access-8kwgj\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.023600 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023359 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 15:12:30.023600 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023364 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-modprobe-d\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.023600 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023491 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-sysconfig\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.023939 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023616 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-kubernetes\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.023939 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023642 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-tuned\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.023939 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023691 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8028e717-3208-45d4-a49f-117180037b85-cni-binary-copy\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.023939 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023721 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-cni-netd\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.023939 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023750 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-system-cni-dir\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.023939 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023775 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8028e717-3208-45d4-a49f-117180037b85-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.023939 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023801 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz48d\" (UniqueName: \"kubernetes.io/projected/8028e717-3208-45d4-a49f-117180037b85-kube-api-access-xz48d\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.023939 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023825 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-env-overrides\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.023939 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023858 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-var-lib-cni-multus\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.023939 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023885 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-lib-modules\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.023939 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023908 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-tmp\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.023939 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023930 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ddf831fb-25bd-459e-8632-8f44d11ecd8b-cni-binary-copy\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023962 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-run-netns\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.023984 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-hostroot\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024006 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024028 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-systemd-units\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024053 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8028e717-3208-45d4-a49f-117180037b85-system-cni-dir\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024085 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8028e717-3208-45d4-a49f-117180037b85-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024109 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs\") pod \"network-metrics-daemon-n5624\" (UID: \"7b189e73-6f40-467e-9bf1-5505c5a7194c\") " pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024131 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024152 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-ovnkube-script-lib\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024182 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkkr8\" (UniqueName: \"kubernetes.io/projected/ed2fe790-e71b-4c4f-885f-0a5661328a04-kube-api-access-xkkr8\") pod \"iptables-alerter-65k22\" (UID: \"ed2fe790-e71b-4c4f-885f-0a5661328a04\") " pod="openshift-network-operator/iptables-alerter-65k22" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024204 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-slash\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024226 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-etc-openvswitch\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024247 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-run-ovn\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024269 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-ovnkube-config\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024305 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sw95\" (UniqueName: \"kubernetes.io/projected/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-kube-api-access-2sw95\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.024509 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024329 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-var-lib-cni-bin\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024352 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8028e717-3208-45d4-a49f-117180037b85-os-release\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024374 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7cff11e5-8f10-4219-9ed9-76a3c70061c8-dbus\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024396 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-ovn-node-metrics-cert\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024416 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-multus-cni-dir\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024470 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-kubelet\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024551 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-run-ovn-kubernetes\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024576 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-var-lib-kubelet\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024603 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-cnibin\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024628 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-run-multus-certs\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024660 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrxqw\" (UniqueName: \"kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw\") pod \"network-check-target-ltmrf\" (UID: \"5354f3d7-6d0a-48bf-b8c1-85949c45d9d2\") " pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024682 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-node-log\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024715 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-log-socket\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024757 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hngkw\" (UniqueName: \"kubernetes.io/projected/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-kube-api-access-hngkw\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024779 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-sysctl-d\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024803 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-sys-fs\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.025231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024820 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8322eaa3-685b-44b5-834f-ae04b6ec6566-host\") pod \"node-ca-k57jv\" (UID: \"8322eaa3-685b-44b5-834f-ae04b6ec6566\") " pod="openshift-image-registry/node-ca-k57jv" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024843 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8028e717-3208-45d4-a49f-117180037b85-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024867 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-run-systemd\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024897 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ddf831fb-25bd-459e-8632-8f44d11ecd8b-multus-daemon-config\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024928 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-registration-dir\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.024998 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ed2fe790-e71b-4c4f-885f-0a5661328a04-iptables-alerter-script\") pod \"iptables-alerter-65k22\" (UID: \"ed2fe790-e71b-4c4f-885f-0a5661328a04\") " pod="openshift-network-operator/iptables-alerter-65k22" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025054 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-cni-bin\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025123 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-run\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025165 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-multus-socket-dir-parent\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025196 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8322eaa3-685b-44b5-834f-ae04b6ec6566-serviceca\") pod \"node-ca-k57jv\" (UID: \"8322eaa3-685b-44b5-834f-ae04b6ec6566\") " pod="openshift-image-registry/node-ca-k57jv" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025222 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7cff11e5-8f10-4219-9ed9-76a3c70061c8-kubelet-config\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025243 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-var-lib-openvswitch\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025265 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-run-openvswitch\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025288 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e73fdf83-f791-4a9b-ae02-4df8329352a2-konnectivity-ca\") pod \"konnectivity-agent-vnw4c\" (UID: \"e73fdf83-f791-4a9b-ae02-4df8329352a2\") " pod="kube-system/konnectivity-agent-vnw4c" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025334 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-var-lib-kubelet\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025356 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-systemd\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.025982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025376 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-sys\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.026809 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025405 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-host\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.026809 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025427 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-run-netns\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.026809 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025455 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e73fdf83-f791-4a9b-ae02-4df8329352a2-agent-certs\") pod \"konnectivity-agent-vnw4c\" (UID: \"e73fdf83-f791-4a9b-ae02-4df8329352a2\") " pod="kube-system/konnectivity-agent-vnw4c" Jul 23 15:12:30.026809 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.025484 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjj65\" (UniqueName: \"kubernetes.io/projected/ddf831fb-25bd-459e-8632-8f44d11ecd8b-kube-api-access-vjj65\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.035865 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.035842 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 15:12:30.036079 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.036064 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 15:12:30.038984 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.038962 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 15:12:30.040549 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.040527 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nfnt7\"" Jul 23 15:12:30.044834 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.044815 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 15:12:30.050401 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.050375 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 15:12:30.058824 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.058788 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zhl2h" Jul 23 15:12:30.068618 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.068547 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zhl2h" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125746 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tns7v\" (UniqueName: \"kubernetes.io/projected/7b189e73-6f40-467e-9bf1-5505c5a7194c-kube-api-access-tns7v\") pod \"network-metrics-daemon-n5624\" (UID: \"7b189e73-6f40-467e-9bf1-5505c5a7194c\") " pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125778 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-socket-dir\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125797 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qj62\" (UniqueName: \"kubernetes.io/projected/8322eaa3-685b-44b5-834f-ae04b6ec6566-kube-api-access-8qj62\") pod \"node-ca-k57jv\" (UID: \"8322eaa3-685b-44b5-834f-ae04b6ec6566\") " pod="openshift-image-registry/node-ca-k57jv" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125818 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-os-release\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125833 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-run-k8s-cni-cncf-io\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125848 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-multus-conf-dir\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125874 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-etc-kubernetes\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125889 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kwgj\" (UniqueName: \"kubernetes.io/projected/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-kube-api-access-8kwgj\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125903 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-modprobe-d\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125917 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-sysconfig\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125931 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-kubernetes\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125945 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-tuned\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125960 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8028e717-3208-45d4-a49f-117180037b85-cni-binary-copy\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125975 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-cni-netd\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.125988 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-system-cni-dir\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126003 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8028e717-3208-45d4-a49f-117180037b85-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.127323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126028 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz48d\" (UniqueName: \"kubernetes.io/projected/8028e717-3208-45d4-a49f-117180037b85-kube-api-access-xz48d\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126044 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-env-overrides\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126059 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-var-lib-cni-multus\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126075 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-lib-modules\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126089 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-tmp\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126103 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ddf831fb-25bd-459e-8632-8f44d11ecd8b-cni-binary-copy\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126117 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-run-netns\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126133 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-hostroot\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126149 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126164 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-systemd-units\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126178 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8028e717-3208-45d4-a49f-117180037b85-system-cni-dir\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126193 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8028e717-3208-45d4-a49f-117180037b85-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126214 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs\") pod \"network-metrics-daemon-n5624\" (UID: \"7b189e73-6f40-467e-9bf1-5505c5a7194c\") " pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126229 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126251 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-ovnkube-script-lib\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126276 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkkr8\" (UniqueName: \"kubernetes.io/projected/ed2fe790-e71b-4c4f-885f-0a5661328a04-kube-api-access-xkkr8\") pod \"iptables-alerter-65k22\" (UID: \"ed2fe790-e71b-4c4f-885f-0a5661328a04\") " pod="openshift-network-operator/iptables-alerter-65k22" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126374 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-system-cni-dir\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.128424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126426 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-slash\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126459 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-etc-openvswitch\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126481 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-etc-kubernetes\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126486 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-run-ovn\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126510 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-ovnkube-config\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126538 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sw95\" (UniqueName: \"kubernetes.io/projected/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-kube-api-access-2sw95\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126563 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-var-lib-cni-bin\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126588 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8028e717-3208-45d4-a49f-117180037b85-os-release\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126613 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7cff11e5-8f10-4219-9ed9-76a3c70061c8-dbus\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126636 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-ovn-node-metrics-cert\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126918 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-modprobe-d\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126975 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-sysconfig\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.126990 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-socket-dir\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.127022 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-kubernetes\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.127139 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-env-overrides\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.127239 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-run-k8s-cni-cncf-io\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.127269 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-os-release\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.127306 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-multus-conf-dir\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.129365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.127325 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.127570 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-systemd-units\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.127652 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-run-netns\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.127682 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-lib-modules\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.127726 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-hostroot\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.127839 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.127912 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret podName:7cff11e5-8f10-4219-9ed9-76a3c70061c8 nodeName:}" failed. No retries permitted until 2026-07-23 15:12:30.627890831 +0000 UTC m=+4.243938076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret") pod "global-pull-secret-syncer-wl9qs" (UID: "7cff11e5-8f10-4219-9ed9-76a3c70061c8") : object "kube-system"/"original-pull-secret" not registered Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.127970 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128022 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8028e717-3208-45d4-a49f-117180037b85-system-cni-dir\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128587 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ddf831fb-25bd-459e-8632-8f44d11ecd8b-cni-binary-copy\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128587 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8028e717-3208-45d4-a49f-117180037b85-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128638 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-multus-cni-dir\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128665 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-kubelet\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.128676 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128691 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-run-ovn-kubernetes\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.128723 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs podName:7b189e73-6f40-467e-9bf1-5505c5a7194c nodeName:}" failed. No retries permitted until 2026-07-23 15:12:30.628706745 +0000 UTC m=+4.244753999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs") pod "network-metrics-daemon-n5624" (UID: "7b189e73-6f40-467e-9bf1-5505c5a7194c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128744 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-var-lib-kubelet\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.130248 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128767 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-cnibin\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128788 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-run-multus-certs\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128809 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrxqw\" (UniqueName: \"kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw\") pod \"network-check-target-ltmrf\" (UID: \"5354f3d7-6d0a-48bf-b8c1-85949c45d9d2\") " pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128831 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-node-log\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128851 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-log-socket\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128874 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hngkw\" (UniqueName: \"kubernetes.io/projected/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-kube-api-access-hngkw\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128897 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-multus-cni-dir\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128933 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-kubelet\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128970 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-run-ovn-kubernetes\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128979 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-sysctl-d\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129039 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-var-lib-kubelet\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129085 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-cnibin\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129123 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-run-multus-certs\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129361 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-node-log\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129404 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-log-socket\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129490 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-ovnkube-script-lib\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.128899 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-sysctl-d\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129537 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-sys-fs\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.131062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129560 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8322eaa3-685b-44b5-834f-ae04b6ec6566-host\") pod \"node-ca-k57jv\" (UID: \"8322eaa3-685b-44b5-834f-ae04b6ec6566\") " pod="openshift-image-registry/node-ca-k57jv" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129586 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8028e717-3208-45d4-a49f-117180037b85-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129615 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-run-systemd\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129618 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-sys-fs\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129641 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ddf831fb-25bd-459e-8632-8f44d11ecd8b-multus-daemon-config\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129657 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8322eaa3-685b-44b5-834f-ae04b6ec6566-host\") pod \"node-ca-k57jv\" (UID: \"8322eaa3-685b-44b5-834f-ae04b6ec6566\") " pod="openshift-image-registry/node-ca-k57jv" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129665 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-registration-dir\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129692 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ed2fe790-e71b-4c4f-885f-0a5661328a04-iptables-alerter-script\") pod \"iptables-alerter-65k22\" (UID: \"ed2fe790-e71b-4c4f-885f-0a5661328a04\") " pod="openshift-network-operator/iptables-alerter-65k22" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129711 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-run-systemd\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129718 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-cni-bin\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129744 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-run\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129769 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-multus-socket-dir-parent\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129792 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8322eaa3-685b-44b5-834f-ae04b6ec6566-serviceca\") pod \"node-ca-k57jv\" (UID: \"8322eaa3-685b-44b5-834f-ae04b6ec6566\") " pod="openshift-image-registry/node-ca-k57jv" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129816 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7cff11e5-8f10-4219-9ed9-76a3c70061c8-kubelet-config\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129843 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-var-lib-openvswitch\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129865 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-run-openvswitch\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129893 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e73fdf83-f791-4a9b-ae02-4df8329352a2-konnectivity-ca\") pod \"konnectivity-agent-vnw4c\" (UID: \"e73fdf83-f791-4a9b-ae02-4df8329352a2\") " pod="kube-system/konnectivity-agent-vnw4c" Jul 23 15:12:30.131942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129919 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-var-lib-kubelet\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129943 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-systemd\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129967 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-sys\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.129990 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-host\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130027 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-run-netns\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130052 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e73fdf83-f791-4a9b-ae02-4df8329352a2-agent-certs\") pod \"konnectivity-agent-vnw4c\" (UID: \"e73fdf83-f791-4a9b-ae02-4df8329352a2\") " pod="kube-system/konnectivity-agent-vnw4c" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130077 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjj65\" (UniqueName: \"kubernetes.io/projected/ddf831fb-25bd-459e-8632-8f44d11ecd8b-kube-api-access-vjj65\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130104 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-etc-selinux\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130129 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8028e717-3208-45d4-a49f-117180037b85-cnibin\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130157 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130184 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-sysctl-conf\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130211 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-device-dir\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130239 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed2fe790-e71b-4c4f-885f-0a5661328a04-host-slash\") pod \"iptables-alerter-65k22\" (UID: \"ed2fe790-e71b-4c4f-885f-0a5661328a04\") " pod="openshift-network-operator/iptables-alerter-65k22" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130356 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed2fe790-e71b-4c4f-885f-0a5661328a04-host-slash\") pod \"iptables-alerter-65k22\" (UID: \"ed2fe790-e71b-4c4f-885f-0a5661328a04\") " pod="openshift-network-operator/iptables-alerter-65k22" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130474 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-var-lib-cni-multus\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130659 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-ovnkube-config\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130773 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ddf831fb-25bd-459e-8632-8f44d11ecd8b-multus-daemon-config\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.132763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130832 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-registration-dir\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131102 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8028e717-3208-45d4-a49f-117180037b85-cni-binary-copy\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131115 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-sys\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131162 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-slash\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131167 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-cni-netd\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131207 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-run-ovn\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131209 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-etc-openvswitch\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131254 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-cni-bin\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131269 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-var-lib-openvswitch\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131326 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-run-openvswitch\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131341 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-tmp\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131360 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ed2fe790-e71b-4c4f-885f-0a5661328a04-iptables-alerter-script\") pod \"iptables-alerter-65k22\" (UID: \"ed2fe790-e71b-4c4f-885f-0a5661328a04\") " pod="openshift-network-operator/iptables-alerter-65k22" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131372 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-run\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131420 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-host\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131425 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-multus-socket-dir-parent\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131431 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-var-lib-kubelet\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131474 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-run-netns\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131487 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7cff11e5-8f10-4219-9ed9-76a3c70061c8-kubelet-config\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:30.133574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131536 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-systemd\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.134437 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131578 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.134437 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131776 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e73fdf83-f791-4a9b-ae02-4df8329352a2-konnectivity-ca\") pod \"konnectivity-agent-vnw4c\" (UID: \"e73fdf83-f791-4a9b-ae02-4df8329352a2\") " pod="kube-system/konnectivity-agent-vnw4c" Jul 23 15:12:30.134437 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131777 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ddf831fb-25bd-459e-8632-8f44d11ecd8b-host-var-lib-cni-bin\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.134437 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131826 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-etc-selinux\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.134437 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.130155 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8028e717-3208-45d4-a49f-117180037b85-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.134437 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131846 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8028e717-3208-45d4-a49f-117180037b85-cnibin\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.134437 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131893 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8322eaa3-685b-44b5-834f-ae04b6ec6566-serviceca\") pod \"node-ca-k57jv\" (UID: \"8322eaa3-685b-44b5-834f-ae04b6ec6566\") " pod="openshift-image-registry/node-ca-k57jv" Jul 23 15:12:30.134437 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131901 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-device-dir\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.134437 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131907 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8028e717-3208-45d4-a49f-117180037b85-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.134437 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131948 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-sysctl-conf\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.134437 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131962 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8028e717-3208-45d4-a49f-117180037b85-os-release\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.134437 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.131988 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7cff11e5-8f10-4219-9ed9-76a3c70061c8-dbus\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:30.134437 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.133158 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-ovn-node-metrics-cert\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.134437 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.133428 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-etc-tuned\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.136289 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.136245 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e73fdf83-f791-4a9b-ae02-4df8329352a2-agent-certs\") pod \"konnectivity-agent-vnw4c\" (UID: \"e73fdf83-f791-4a9b-ae02-4df8329352a2\") " pod="kube-system/konnectivity-agent-vnw4c" Jul 23 15:12:30.143550 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.143488 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkkr8\" (UniqueName: \"kubernetes.io/projected/ed2fe790-e71b-4c4f-885f-0a5661328a04-kube-api-access-xkkr8\") pod \"iptables-alerter-65k22\" (UID: \"ed2fe790-e71b-4c4f-885f-0a5661328a04\") " pod="openshift-network-operator/iptables-alerter-65k22" Jul 23 15:12:30.152937 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.152772 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hngkw\" (UniqueName: \"kubernetes.io/projected/946dd7b0-9ff6-41fe-9515-3e1d5b2e3867-kube-api-access-hngkw\") pod \"ovnkube-node-cdjsd\" (UID: \"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867\") " pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.155760 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.155734 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kwgj\" (UniqueName: \"kubernetes.io/projected/f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285-kube-api-access-8kwgj\") pod \"aws-ebs-csi-driver-node-9qtxv\" (UID: \"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.155901 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.155880 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjj65\" (UniqueName: \"kubernetes.io/projected/ddf831fb-25bd-459e-8632-8f44d11ecd8b-kube-api-access-vjj65\") pod \"multus-rdgrb\" (UID: \"ddf831fb-25bd-459e-8632-8f44d11ecd8b\") " pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.157489 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.157424 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 15:12:30.157489 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.157451 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 15:12:30.157489 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.157465 2570 projected.go:196] Error preparing data for projected volume kube-api-access-zrxqw for pod openshift-network-diagnostics/network-check-target-ltmrf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:12:30.157705 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.157541 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw podName:5354f3d7-6d0a-48bf-b8c1-85949c45d9d2 nodeName:}" failed. No retries permitted until 2026-07-23 15:12:30.657522928 +0000 UTC m=+4.273570187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zrxqw" (UniqueName: "kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw") pod "network-check-target-ltmrf" (UID: "5354f3d7-6d0a-48bf-b8c1-85949c45d9d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:12:30.159947 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.159925 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qj62\" (UniqueName: \"kubernetes.io/projected/8322eaa3-685b-44b5-834f-ae04b6ec6566-kube-api-access-8qj62\") pod \"node-ca-k57jv\" (UID: \"8322eaa3-685b-44b5-834f-ae04b6ec6566\") " pod="openshift-image-registry/node-ca-k57jv" Jul 23 15:12:30.160262 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.160239 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sw95\" (UniqueName: \"kubernetes.io/projected/3cf085a6-b6c8-4e60-8cd5-450c7c8e006f-kube-api-access-2sw95\") pod \"tuned-njgnh\" (UID: \"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f\") " pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.165551 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.165504 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tns7v\" (UniqueName: \"kubernetes.io/projected/7b189e73-6f40-467e-9bf1-5505c5a7194c-kube-api-access-tns7v\") pod \"network-metrics-daemon-n5624\" (UID: \"7b189e73-6f40-467e-9bf1-5505c5a7194c\") " pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:30.165658 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.165586 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz48d\" (UniqueName: \"kubernetes.io/projected/8028e717-3208-45d4-a49f-117180037b85-kube-api-access-xz48d\") pod \"multus-additional-cni-plugins-4bhpc\" (UID: \"8028e717-3208-45d4-a49f-117180037b85\") " pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.314330 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.314034 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vnw4c" Jul 23 15:12:30.324705 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.324507 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-njgnh" Jul 23 15:12:30.325520 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:30.325490 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode73fdf83_f791_4a9b_ae02_4df8329352a2.slice/crio-64c0570b344b114e0b382aa47e1da837c5168a571352d39f8934d1181077b0e5 WatchSource:0}: Error finding container 64c0570b344b114e0b382aa47e1da837c5168a571352d39f8934d1181077b0e5: Status 404 returned error can't find the container with id 64c0570b344b114e0b382aa47e1da837c5168a571352d39f8934d1181077b0e5 Jul 23 15:12:30.334255 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.334028 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k57jv" Jul 23 15:12:30.336732 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:30.336704 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf085a6_b6c8_4e60_8cd5_450c7c8e006f.slice/crio-e956d159c03a19da28df24dd2b320cc17fbadc5cdca9dea52b713c760a6cb62f WatchSource:0}: Error finding container e956d159c03a19da28df24dd2b320cc17fbadc5cdca9dea52b713c760a6cb62f: Status 404 returned error can't find the container with id e956d159c03a19da28df24dd2b320cc17fbadc5cdca9dea52b713c760a6cb62f Jul 23 15:12:30.342589 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.342395 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4bhpc" Jul 23 15:12:30.345335 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:30.345307 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8322eaa3_685b_44b5_834f_ae04b6ec6566.slice/crio-3008e2ba552e1c61cddd65c480ef04a351c574c75852389fe6cf9396d6aadca9 WatchSource:0}: Error finding container 3008e2ba552e1c61cddd65c480ef04a351c574c75852389fe6cf9396d6aadca9: Status 404 returned error can't find the container with id 3008e2ba552e1c61cddd65c480ef04a351c574c75852389fe6cf9396d6aadca9 Jul 23 15:12:30.352653 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:30.352632 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8028e717_3208_45d4_a49f_117180037b85.slice/crio-f40fd726553eae9328a57d81b0458de88489a19f7c2b07423a2a91f1145c70e1 WatchSource:0}: Error finding container f40fd726553eae9328a57d81b0458de88489a19f7c2b07423a2a91f1145c70e1: Status 404 returned error can't find the container with id f40fd726553eae9328a57d81b0458de88489a19f7c2b07423a2a91f1145c70e1 Jul 23 15:12:30.355035 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.355014 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rdgrb" Jul 23 15:12:30.363587 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.363427 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" Jul 23 15:12:30.364550 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:30.364527 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf831fb_25bd_459e_8632_8f44d11ecd8b.slice/crio-b24f8248db1b11761bb6f3b3c95cebb5a2825cfab94fa0267b50d24b8ea65e78 WatchSource:0}: Error finding container b24f8248db1b11761bb6f3b3c95cebb5a2825cfab94fa0267b50d24b8ea65e78: Status 404 returned error can't find the container with id b24f8248db1b11761bb6f3b3c95cebb5a2825cfab94fa0267b50d24b8ea65e78 Jul 23 15:12:30.374370 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.374096 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-65k22" Jul 23 15:12:30.374734 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:30.374651 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8ea8b30_c2ef_4bfc_a438_46e0fdcd0285.slice/crio-5874a93421b4520249c4735d374a06574d35ee3c7fac8fbec4fe6f727b07ef28 WatchSource:0}: Error finding container 5874a93421b4520249c4735d374a06574d35ee3c7fac8fbec4fe6f727b07ef28: Status 404 returned error can't find the container with id 5874a93421b4520249c4735d374a06574d35ee3c7fac8fbec4fe6f727b07ef28 Jul 23 15:12:30.381929 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.381909 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:30.386337 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:30.386283 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded2fe790_e71b_4c4f_885f_0a5661328a04.slice/crio-d3c84d1ee140aaab54dd7f32fd4bac7279b6370d1bc6ab9b69ac901d03e3d719 WatchSource:0}: Error finding container d3c84d1ee140aaab54dd7f32fd4bac7279b6370d1bc6ab9b69ac901d03e3d719: Status 404 returned error can't find the container with id d3c84d1ee140aaab54dd7f32fd4bac7279b6370d1bc6ab9b69ac901d03e3d719 Jul 23 15:12:30.393190 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:30.393166 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod946dd7b0_9ff6_41fe_9515_3e1d5b2e3867.slice/crio-8fc3d97ddd8ea683d367b440a8b20af5744619ff18ca2b129f4d7ae2af80b084 WatchSource:0}: Error finding container 8fc3d97ddd8ea683d367b440a8b20af5744619ff18ca2b129f4d7ae2af80b084: Status 404 returned error can't find the container with id 8fc3d97ddd8ea683d367b440a8b20af5744619ff18ca2b129f4d7ae2af80b084 Jul 23 15:12:30.420259 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.420230 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 15:12:30.437582 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.437523 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-njgnh" event={"ID":"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f","Type":"ContainerStarted","Data":"e956d159c03a19da28df24dd2b320cc17fbadc5cdca9dea52b713c760a6cb62f"} Jul 23 15:12:30.439036 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.439002 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-63.ec2.internal" event={"ID":"b932a6350e4894a972718b80947507d2","Type":"ContainerStarted","Data":"560b559c1f341c2766eceba5736d5c9f5dbdd5356841b731c21f69cb5ffaa79d"} Jul 23 15:12:30.440388 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.440359 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" event={"ID":"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867","Type":"ContainerStarted","Data":"8fc3d97ddd8ea683d367b440a8b20af5744619ff18ca2b129f4d7ae2af80b084"} Jul 23 15:12:30.441806 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.441780 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" event={"ID":"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285","Type":"ContainerStarted","Data":"5874a93421b4520249c4735d374a06574d35ee3c7fac8fbec4fe6f727b07ef28"} Jul 23 15:12:30.443204 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.443173 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bhpc" event={"ID":"8028e717-3208-45d4-a49f-117180037b85","Type":"ContainerStarted","Data":"f40fd726553eae9328a57d81b0458de88489a19f7c2b07423a2a91f1145c70e1"} Jul 23 15:12:30.444600 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.444571 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vnw4c" event={"ID":"e73fdf83-f791-4a9b-ae02-4df8329352a2","Type":"ContainerStarted","Data":"64c0570b344b114e0b382aa47e1da837c5168a571352d39f8934d1181077b0e5"} Jul 23 15:12:30.445728 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.445693 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal" event={"ID":"67b7466e3c1de219c8d3784cc786446a","Type":"ContainerStarted","Data":"e33b8c28a9de75e7a9d4050cefe2ad36955a5cc652775faf54abc49ec3c00ce8"} Jul 23 15:12:30.447006 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.446979 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-65k22" event={"ID":"ed2fe790-e71b-4c4f-885f-0a5661328a04","Type":"ContainerStarted","Data":"d3c84d1ee140aaab54dd7f32fd4bac7279b6370d1bc6ab9b69ac901d03e3d719"} Jul 23 15:12:30.448362 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.448339 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rdgrb" event={"ID":"ddf831fb-25bd-459e-8632-8f44d11ecd8b","Type":"ContainerStarted","Data":"b24f8248db1b11761bb6f3b3c95cebb5a2825cfab94fa0267b50d24b8ea65e78"} Jul 23 15:12:30.449638 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.449616 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k57jv" event={"ID":"8322eaa3-685b-44b5-834f-ae04b6ec6566","Type":"ContainerStarted","Data":"3008e2ba552e1c61cddd65c480ef04a351c574c75852389fe6cf9396d6aadca9"} Jul 23 15:12:30.632757 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.632677 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:30.632757 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.632710 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs\") pod \"network-metrics-daemon-n5624\" (UID: \"7b189e73-6f40-467e-9bf1-5505c5a7194c\") " pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:30.632963 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.632848 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 15:12:30.632963 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.632921 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret podName:7cff11e5-8f10-4219-9ed9-76a3c70061c8 nodeName:}" failed. No retries permitted until 2026-07-23 15:12:31.632902377 +0000 UTC m=+5.248949639 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret") pod "global-pull-secret-syncer-wl9qs" (UID: "7cff11e5-8f10-4219-9ed9-76a3c70061c8") : object "kube-system"/"original-pull-secret" not registered Jul 23 15:12:30.633063 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.632852 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:12:30.633063 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.633020 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs podName:7b189e73-6f40-467e-9bf1-5505c5a7194c nodeName:}" failed. No retries permitted until 2026-07-23 15:12:31.633005822 +0000 UTC m=+5.249053071 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs") pod "network-metrics-daemon-n5624" (UID: "7b189e73-6f40-467e-9bf1-5505c5a7194c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:12:30.733813 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:30.733770 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrxqw\" (UniqueName: \"kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw\") pod \"network-check-target-ltmrf\" (UID: \"5354f3d7-6d0a-48bf-b8c1-85949c45d9d2\") " pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:30.734008 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.733931 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 15:12:30.734008 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.733958 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 15:12:30.734008 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.733976 2570 projected.go:196] Error preparing data for projected volume kube-api-access-zrxqw for pod openshift-network-diagnostics/network-check-target-ltmrf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:12:30.734170 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:30.734049 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw podName:5354f3d7-6d0a-48bf-b8c1-85949c45d9d2 nodeName:}" failed. No retries permitted until 2026-07-23 15:12:31.7340286 +0000 UTC m=+5.350075862 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zrxqw" (UniqueName: "kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw") pod "network-check-target-ltmrf" (UID: "5354f3d7-6d0a-48bf-b8c1-85949c45d9d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:12:31.071503 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:31.071456 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 15:07:30 +0000 UTC" deadline="2028-01-30 04:28:53.298866262 +0000 UTC" Jul 23 15:12:31.071503 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:31.071500 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13333h16m22.227370835s" Jul 23 15:12:31.433022 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:31.432745 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:31.433022 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:31.432885 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:31.433332 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:31.433308 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:31.433427 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:31.433404 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:31.433486 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:31.433472 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:31.433564 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:31.433544 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:31.641377 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:31.641339 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:31.641572 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:31.641394 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs\") pod \"network-metrics-daemon-n5624\" (UID: \"7b189e73-6f40-467e-9bf1-5505c5a7194c\") " pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:31.641641 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:31.641568 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:12:31.641641 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:31.641631 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs podName:7b189e73-6f40-467e-9bf1-5505c5a7194c nodeName:}" failed. No retries permitted until 2026-07-23 15:12:33.641613838 +0000 UTC m=+7.257661083 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs") pod "network-metrics-daemon-n5624" (UID: "7b189e73-6f40-467e-9bf1-5505c5a7194c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:12:31.642044 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:31.642024 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 15:12:31.642144 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:31.642079 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret podName:7cff11e5-8f10-4219-9ed9-76a3c70061c8 nodeName:}" failed. No retries permitted until 2026-07-23 15:12:33.642062777 +0000 UTC m=+7.258110021 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret") pod "global-pull-secret-syncer-wl9qs" (UID: "7cff11e5-8f10-4219-9ed9-76a3c70061c8") : object "kube-system"/"original-pull-secret" not registered Jul 23 15:12:31.742641 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:31.742554 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrxqw\" (UniqueName: \"kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw\") pod \"network-check-target-ltmrf\" (UID: \"5354f3d7-6d0a-48bf-b8c1-85949c45d9d2\") " pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:31.742817 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:31.742756 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 15:12:31.742817 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:31.742789 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 15:12:31.742817 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:31.742803 2570 projected.go:196] Error preparing data for projected volume kube-api-access-zrxqw for pod openshift-network-diagnostics/network-check-target-ltmrf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:12:31.742971 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:31.742864 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw podName:5354f3d7-6d0a-48bf-b8c1-85949c45d9d2 nodeName:}" failed. No retries permitted until 2026-07-23 15:12:33.742844455 +0000 UTC m=+7.358891710 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zrxqw" (UniqueName: "kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw") pod "network-check-target-ltmrf" (UID: "5354f3d7-6d0a-48bf-b8c1-85949c45d9d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:12:32.072756 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:32.072620 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 15:07:30 +0000 UTC" deadline="2027-12-24 12:30:46.70821805 +0000 UTC" Jul 23 15:12:32.072756 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:32.072656 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12453h18m14.63556599s" Jul 23 15:12:32.480513 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:32.480245 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-63.ec2.internal" event={"ID":"b932a6350e4894a972718b80947507d2","Type":"ContainerStarted","Data":"2abdddb55ac4d4b2a7a8923d62af36884ece4dc3b56efbeb08ba5748bf038546"} Jul 23 15:12:33.432862 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:33.432821 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:33.433361 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:33.432971 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:33.433436 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:33.433421 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:33.433541 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:33.433517 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:33.433594 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:33.433587 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:33.433678 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:33.433658 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:33.485508 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:33.485467 2570 generic.go:356] "Generic (PLEG): container finished" podID="67b7466e3c1de219c8d3784cc786446a" containerID="2cdbfb164871d786f3dd845688e53875efbad584e2f2147b0dea073f90eaf009" exitCode=0 Jul 23 15:12:33.486704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:33.486419 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal" event={"ID":"67b7466e3c1de219c8d3784cc786446a","Type":"ContainerDied","Data":"2cdbfb164871d786f3dd845688e53875efbad584e2f2147b0dea073f90eaf009"} Jul 23 15:12:33.509459 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:33.508237 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-63.ec2.internal" podStartSLOduration=4.508213951 podStartE2EDuration="4.508213951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:12:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:12:32.498113913 +0000 UTC m=+6.114161178" watchObservedRunningTime="2026-07-23 15:12:33.508213951 +0000 UTC m=+7.124261202" Jul 23 15:12:33.659267 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:33.659222 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:33.659467 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:33.659281 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs\") pod \"network-metrics-daemon-n5624\" (UID: \"7b189e73-6f40-467e-9bf1-5505c5a7194c\") " pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:33.659467 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:33.659418 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:12:33.659580 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:33.659480 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs podName:7b189e73-6f40-467e-9bf1-5505c5a7194c nodeName:}" failed. No retries permitted until 2026-07-23 15:12:37.659460672 +0000 UTC m=+11.275507921 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs") pod "network-metrics-daemon-n5624" (UID: "7b189e73-6f40-467e-9bf1-5505c5a7194c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:12:33.659580 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:33.659552 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 15:12:33.659684 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:33.659585 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret podName:7cff11e5-8f10-4219-9ed9-76a3c70061c8 nodeName:}" failed. No retries permitted until 2026-07-23 15:12:37.659574243 +0000 UTC m=+11.275621501 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret") pod "global-pull-secret-syncer-wl9qs" (UID: "7cff11e5-8f10-4219-9ed9-76a3c70061c8") : object "kube-system"/"original-pull-secret" not registered Jul 23 15:12:33.760785 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:33.760701 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrxqw\" (UniqueName: \"kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw\") pod \"network-check-target-ltmrf\" (UID: \"5354f3d7-6d0a-48bf-b8c1-85949c45d9d2\") " pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:33.760950 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:33.760848 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 15:12:33.760950 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:33.760871 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 15:12:33.760950 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:33.760884 2570 projected.go:196] Error preparing data for projected volume kube-api-access-zrxqw for pod openshift-network-diagnostics/network-check-target-ltmrf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:12:33.760950 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:33.760941 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw podName:5354f3d7-6d0a-48bf-b8c1-85949c45d9d2 nodeName:}" failed. No retries permitted until 2026-07-23 15:12:37.760922651 +0000 UTC m=+11.376969894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zrxqw" (UniqueName: "kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw") pod "network-check-target-ltmrf" (UID: "5354f3d7-6d0a-48bf-b8c1-85949c45d9d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:12:35.432873 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:35.432838 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:35.433385 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:35.432991 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:35.433385 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:35.433013 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:35.433385 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:35.433087 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:35.433385 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:35.433130 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:35.433385 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:35.433190 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:37.433019 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:37.432941 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:37.433019 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:37.432970 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:37.433604 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:37.432954 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:37.433604 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:37.433089 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:37.433604 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:37.433159 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:37.433604 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:37.433243 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:37.697753 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:37.697670 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:37.697753 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:37.697732 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs\") pod \"network-metrics-daemon-n5624\" (UID: \"7b189e73-6f40-467e-9bf1-5505c5a7194c\") " pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:37.697979 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:37.697905 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:12:37.697979 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:37.697977 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs podName:7b189e73-6f40-467e-9bf1-5505c5a7194c nodeName:}" failed. No retries permitted until 2026-07-23 15:12:45.69795025 +0000 UTC m=+19.313997495 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs") pod "network-metrics-daemon-n5624" (UID: "7b189e73-6f40-467e-9bf1-5505c5a7194c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:12:37.698485 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:37.698458 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 15:12:37.698613 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:37.698535 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret podName:7cff11e5-8f10-4219-9ed9-76a3c70061c8 nodeName:}" failed. No retries permitted until 2026-07-23 15:12:45.698518825 +0000 UTC m=+19.314566073 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret") pod "global-pull-secret-syncer-wl9qs" (UID: "7cff11e5-8f10-4219-9ed9-76a3c70061c8") : object "kube-system"/"original-pull-secret" not registered Jul 23 15:12:37.798510 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:37.798394 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrxqw\" (UniqueName: \"kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw\") pod \"network-check-target-ltmrf\" (UID: \"5354f3d7-6d0a-48bf-b8c1-85949c45d9d2\") " pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:37.798685 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:37.798560 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 15:12:37.798685 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:37.798581 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 15:12:37.798685 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:37.798594 2570 projected.go:196] Error preparing data for projected volume kube-api-access-zrxqw for pod openshift-network-diagnostics/network-check-target-ltmrf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:12:37.798685 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:37.798660 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw podName:5354f3d7-6d0a-48bf-b8c1-85949c45d9d2 nodeName:}" failed. No retries permitted until 2026-07-23 15:12:45.798642102 +0000 UTC m=+19.414689347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zrxqw" (UniqueName: "kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw") pod "network-check-target-ltmrf" (UID: "5354f3d7-6d0a-48bf-b8c1-85949c45d9d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:12:39.432366 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:39.432327 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:39.432837 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:39.432460 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:39.432837 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:39.432787 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:39.432955 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:39.432875 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:39.432955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:39.432922 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:39.433053 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:39.433007 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:41.432869 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:41.432781 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:41.433339 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:41.432781 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:41.433339 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:41.432935 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:41.433339 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:41.433008 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:41.433339 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:41.432781 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:41.433339 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:41.433122 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:43.432493 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:43.432458 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:43.432925 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:43.432458 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:43.432925 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:43.432566 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:43.432925 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:43.432458 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:43.432925 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:43.432643 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:43.432925 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:43.432697 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:45.432146 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:45.432064 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:45.432146 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:45.432109 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:45.432146 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:45.432109 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:45.432791 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:45.432203 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:45.432791 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:45.432279 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:45.432791 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:45.432371 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:45.758820 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:45.758745 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:45.758820 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:45.758780 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs\") pod \"network-metrics-daemon-n5624\" (UID: \"7b189e73-6f40-467e-9bf1-5505c5a7194c\") " pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:45.759005 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:45.758886 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:12:45.759005 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:45.758936 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs podName:7b189e73-6f40-467e-9bf1-5505c5a7194c nodeName:}" failed. No retries permitted until 2026-07-23 15:13:01.758923126 +0000 UTC m=+35.374970369 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs") pod "network-metrics-daemon-n5624" (UID: "7b189e73-6f40-467e-9bf1-5505c5a7194c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:12:45.759005 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:45.758886 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 15:12:45.759134 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:45.759012 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret podName:7cff11e5-8f10-4219-9ed9-76a3c70061c8 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:01.758999547 +0000 UTC m=+35.375046794 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret") pod "global-pull-secret-syncer-wl9qs" (UID: "7cff11e5-8f10-4219-9ed9-76a3c70061c8") : object "kube-system"/"original-pull-secret" not registered Jul 23 15:12:45.859492 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:45.859460 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrxqw\" (UniqueName: \"kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw\") pod \"network-check-target-ltmrf\" (UID: \"5354f3d7-6d0a-48bf-b8c1-85949c45d9d2\") " pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:45.859641 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:45.859610 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 15:12:45.859641 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:45.859630 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 15:12:45.859641 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:45.859640 2570 projected.go:196] Error preparing data for projected volume kube-api-access-zrxqw for pod openshift-network-diagnostics/network-check-target-ltmrf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:12:45.859765 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:45.859704 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw podName:5354f3d7-6d0a-48bf-b8c1-85949c45d9d2 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:01.859682747 +0000 UTC m=+35.475729994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zrxqw" (UniqueName: "kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw") pod "network-check-target-ltmrf" (UID: "5354f3d7-6d0a-48bf-b8c1-85949c45d9d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:12:47.432275 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:47.432229 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:47.432734 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:47.432290 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:47.432734 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:47.432239 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:47.432734 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:47.432423 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:47.432734 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:47.432530 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:47.432734 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:47.432645 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:49.433005 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.432844 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:49.433371 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.432965 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:49.433371 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:49.433111 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:49.433371 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:49.433203 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:49.433371 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.432997 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:49.433371 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:49.433316 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:49.520042 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.520007 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k57jv" event={"ID":"8322eaa3-685b-44b5-834f-ae04b6ec6566","Type":"ContainerStarted","Data":"eeb186dfc4faae7f874cd4f685b90d1ecd648d00a4cfbdc813c1be342e0d900c"} Jul 23 15:12:49.524313 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.521861 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-njgnh" event={"ID":"3cf085a6-b6c8-4e60-8cd5-450c7c8e006f","Type":"ContainerStarted","Data":"39278e7dc000dfabfb23d65f7bdd585b1733b25dc85bb995dc4ed4483ee29281"} Jul 23 15:12:49.524313 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.523587 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" event={"ID":"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285","Type":"ContainerStarted","Data":"3bb9b7f36273948819c68bbec670c2d3a1f18e21077f546b395e683ed696433b"} Jul 23 15:12:49.525815 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.525791 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bhpc" event={"ID":"8028e717-3208-45d4-a49f-117180037b85","Type":"ContainerStarted","Data":"b587974bd2ade1fa6375932995e3296e391fa35cd8a51f7fa5d4b9afa436d880"} Jul 23 15:12:49.527042 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.527023 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vnw4c" event={"ID":"e73fdf83-f791-4a9b-ae02-4df8329352a2","Type":"ContainerStarted","Data":"a31c437741c41a9b84decdbd16f2443cf083a75a63485ce70e72174ef9f870b6"} Jul 23 15:12:49.528679 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.528661 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal" event={"ID":"67b7466e3c1de219c8d3784cc786446a","Type":"ContainerStarted","Data":"5e51f7b4094ab685de90104a3c9d7260b361c94dad8286ff707f1be0a214ff8a"} Jul 23 15:12:49.529867 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.529849 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rdgrb" event={"ID":"ddf831fb-25bd-459e-8632-8f44d11ecd8b","Type":"ContainerStarted","Data":"5d0019e27430f3032e8c0bcc6f8976766dad8c618f84f799d12d6e20ed88226d"} Jul 23 15:12:49.557602 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.557558 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-k57jv" podStartSLOduration=2.7283003 podStartE2EDuration="21.557544992s" totalImagesPullingTime="18.829244692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:12:49.557528622 +0000 UTC m=+23.173575885" watchObservedRunningTime="2026-07-23 15:12:49.557544992 +0000 UTC m=+23.173592299" Jul 23 15:12:49.595025 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.594948 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-63.ec2.internal" podStartSLOduration=20.594925829 podStartE2EDuration="20.594925829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:12:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:12:49.573360016 +0000 UTC m=+23.189407280" watchObservedRunningTime="2026-07-23 15:12:49.594925829 +0000 UTC m=+23.210973094" Jul 23 15:12:49.612651 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.612608 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vnw4c" podStartSLOduration=7.265681473 podStartE2EDuration="21.612594206s" totalImagesPullingTime="14.346912733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:12:49.612434601 +0000 UTC m=+23.228481866" watchObservedRunningTime="2026-07-23 15:12:49.612594206 +0000 UTC m=+23.228641470" Jul 23 15:12:49.612910 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.612888 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-njgnh" podStartSLOduration=2.755838428 podStartE2EDuration="21.612883305s" totalImagesPullingTime="18.857044877s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:12:49.595090068 +0000 UTC m=+23.211137346" watchObservedRunningTime="2026-07-23 15:12:49.612883305 +0000 UTC m=+23.228930570" Jul 23 15:12:49.678537 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:49.678486 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rdgrb" podStartSLOduration=2.830934782 podStartE2EDuration="21.678470404s" totalImagesPullingTime="18.847535622s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:12:49.678273324 +0000 UTC m=+23.294320588" watchObservedRunningTime="2026-07-23 15:12:49.678470404 +0000 UTC m=+23.294517659" Jul 23 15:12:50.534463 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:50.534262 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cdjsd_946dd7b0-9ff6-41fe-9515-3e1d5b2e3867/ovn-acl-logging/0.log" Jul 23 15:12:50.535123 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:50.534746 2570 generic.go:356] "Generic (PLEG): container finished" podID="946dd7b0-9ff6-41fe-9515-3e1d5b2e3867" containerID="0764dfefce3c8724e9be34132d2e63f29d8463054be482c84928a393e48b37ff" exitCode=1 Jul 23 15:12:50.535123 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:50.534823 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" event={"ID":"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867","Type":"ContainerStarted","Data":"964ebde0c1611b7eaf204a9a65adc204bebcf6239aad704ec662790299686971"} Jul 23 15:12:50.535123 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:50.534907 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" event={"ID":"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867","Type":"ContainerStarted","Data":"c6477a0e5ec88a425ec49219a22232ea19e76ebbfaa09548ef7663453ecf2b19"} Jul 23 15:12:50.535123 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:50.534960 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" event={"ID":"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867","Type":"ContainerStarted","Data":"b282f66505cfe6ecdeb5a7c47e214c82c16eb8f0dac2e3bd7f5e505823841126"} Jul 23 15:12:50.535123 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:50.535008 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" event={"ID":"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867","Type":"ContainerStarted","Data":"bae75a73f9e3c1b365578c6ae1970c5ecbb21cf16a66bad7e6bf877339167ef0"} Jul 23 15:12:50.535123 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:50.535050 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" event={"ID":"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867","Type":"ContainerDied","Data":"0764dfefce3c8724e9be34132d2e63f29d8463054be482c84928a393e48b37ff"} Jul 23 15:12:50.535123 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:50.535095 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" event={"ID":"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867","Type":"ContainerStarted","Data":"c264ffc711242992919806268eea03dd4d8a29de2a2e4616fa033eeaf9c3427c"} Jul 23 15:12:50.536350 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:50.536328 2570 generic.go:356] "Generic (PLEG): container finished" podID="8028e717-3208-45d4-a49f-117180037b85" containerID="b587974bd2ade1fa6375932995e3296e391fa35cd8a51f7fa5d4b9afa436d880" exitCode=0 Jul 23 15:12:50.536461 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:50.536422 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bhpc" event={"ID":"8028e717-3208-45d4-a49f-117180037b85","Type":"ContainerDied","Data":"b587974bd2ade1fa6375932995e3296e391fa35cd8a51f7fa5d4b9afa436d880"} Jul 23 15:12:50.651115 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:50.651095 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 15:12:51.363083 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:51.362895 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T15:12:50.651109563Z","UUID":"f944a051-d834-4358-b858-934d00228514","Handler":null,"Name":"","Endpoint":""} Jul 23 15:12:51.364524 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:51.364503 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 15:12:51.364652 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:51.364546 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 15:12:51.394574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:51.394542 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vnw4c" Jul 23 15:12:51.395137 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:51.395111 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vnw4c" Jul 23 15:12:51.432519 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:51.432435 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:51.432519 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:51.432478 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:51.432785 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:51.432550 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:51.432785 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:51.432564 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:51.432785 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:51.432648 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:51.432785 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:51.432722 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:51.541900 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:51.541763 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-65k22" event={"ID":"ed2fe790-e71b-4c4f-885f-0a5661328a04","Type":"ContainerStarted","Data":"af6f9c2389b0a8da4b21fdc0497af6cc38d303cf57733a1895b0a1a4283846cb"} Jul 23 15:12:51.544539 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:51.544506 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" event={"ID":"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285","Type":"ContainerStarted","Data":"ebcd6e9218126025e4ccf05187890c4d11e62580347c6d61cea1c67de65e2f5f"} Jul 23 15:12:51.544829 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:51.544782 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vnw4c" Jul 23 15:12:51.545362 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:51.545341 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vnw4c" Jul 23 15:12:51.607618 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:51.607574 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-65k22" podStartSLOduration=4.817838632 podStartE2EDuration="23.607556619s" totalImagesPullingTime="18.789717987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:12:51.580328565 +0000 UTC m=+25.196375830" watchObservedRunningTime="2026-07-23 15:12:51.607556619 +0000 UTC m=+25.223603913" Jul 23 15:12:52.549290 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:52.549237 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" event={"ID":"f8ea8b30-c2ef-4bfc-a438-46e0fdcd0285","Type":"ContainerStarted","Data":"21b9241d738b05fd23b9674792496ba855e5f89fa564ade03271ff2cbae7b28f"} Jul 23 15:12:52.571387 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:52.571336 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qtxv" podStartSLOduration=3.444141825 podStartE2EDuration="24.571316478s" totalImagesPullingTime="21.127174653s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:12:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:12:52.571249354 +0000 UTC m=+26.187296619" watchObservedRunningTime="2026-07-23 15:12:52.571316478 +0000 UTC m=+26.187363743" Jul 23 15:12:53.432405 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:53.432193 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:53.432648 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:53.432193 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:53.432648 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:53.432502 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:53.432648 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:53.432597 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:53.432648 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:53.432224 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:53.432869 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:53.432709 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:53.555228 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:53.555190 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cdjsd_946dd7b0-9ff6-41fe-9515-3e1d5b2e3867/ovn-acl-logging/0.log" Jul 23 15:12:53.555669 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:53.555646 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" event={"ID":"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867","Type":"ContainerStarted","Data":"f225f8a8f697ea45212ddd89d008f5a64d70b6fb064a2aaa6f3f5c36146a7fbe"} Jul 23 15:12:55.432685 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:55.432489 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:55.433501 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:55.432534 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:55.433501 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:55.432764 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:55.433501 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:55.432848 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:55.433501 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:55.432534 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:55.433501 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:55.432948 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:55.563458 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:55.563434 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cdjsd_946dd7b0-9ff6-41fe-9515-3e1d5b2e3867/ovn-acl-logging/0.log" Jul 23 15:12:55.563796 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:55.563775 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" event={"ID":"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867","Type":"ContainerStarted","Data":"1677bda1b3efc20b52007e439ba5c5f4a744327e703334a0e077c8e9eb7de27a"} Jul 23 15:12:55.564016 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:55.563994 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:55.564235 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:55.564211 2570 scope.go:117] "RemoveContainer" containerID="0764dfefce3c8724e9be34132d2e63f29d8463054be482c84928a393e48b37ff" Jul 23 15:12:55.565749 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:55.565727 2570 generic.go:356] "Generic (PLEG): container finished" podID="8028e717-3208-45d4-a49f-117180037b85" containerID="78896a806c8eebd65cc6f41d9bdfd8f2ca8025bce21e8153f21c1ce35a404ea7" exitCode=0 Jul 23 15:12:55.565852 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:55.565763 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bhpc" event={"ID":"8028e717-3208-45d4-a49f-117180037b85","Type":"ContainerDied","Data":"78896a806c8eebd65cc6f41d9bdfd8f2ca8025bce21e8153f21c1ce35a404ea7"} Jul 23 15:12:55.580558 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:55.580540 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:56.496158 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.496123 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-mwbjc"] Jul 23 15:12:56.509388 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.509357 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mwbjc" Jul 23 15:12:56.512364 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.512332 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 15:12:56.512364 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.512362 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 15:12:56.513378 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.513364 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gv85d\"" Jul 23 15:12:56.572679 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.572651 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cdjsd_946dd7b0-9ff6-41fe-9515-3e1d5b2e3867/ovn-acl-logging/0.log" Jul 23 15:12:56.573073 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.573044 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" event={"ID":"946dd7b0-9ff6-41fe-9515-3e1d5b2e3867","Type":"ContainerStarted","Data":"65f55be4ecae3e1c5655974144ca64887a553bc68aa6a394cb0daf44646820ab"} Jul 23 15:12:56.573454 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.573429 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:56.573564 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.573500 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:56.589128 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.589097 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:12:56.605860 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.605803 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" podStartSLOduration=9.388117979 podStartE2EDuration="28.60578191s" totalImagesPullingTime="19.217663931s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:12:56.604019426 +0000 UTC m=+30.220066690" watchObservedRunningTime="2026-07-23 15:12:56.60578191 +0000 UTC m=+30.221829175" Jul 23 15:12:56.632075 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.632051 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/15c40f6c-249b-4f48-a8ae-f8b6da077e0e-hosts-file\") pod \"node-resolver-mwbjc\" (UID: \"15c40f6c-249b-4f48-a8ae-f8b6da077e0e\") " pod="openshift-dns/node-resolver-mwbjc" Jul 23 15:12:56.632191 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.632111 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15c40f6c-249b-4f48-a8ae-f8b6da077e0e-tmp-dir\") pod \"node-resolver-mwbjc\" (UID: \"15c40f6c-249b-4f48-a8ae-f8b6da077e0e\") " pod="openshift-dns/node-resolver-mwbjc" Jul 23 15:12:56.632230 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.632206 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg72s\" (UniqueName: \"kubernetes.io/projected/15c40f6c-249b-4f48-a8ae-f8b6da077e0e-kube-api-access-kg72s\") pod \"node-resolver-mwbjc\" (UID: \"15c40f6c-249b-4f48-a8ae-f8b6da077e0e\") " pod="openshift-dns/node-resolver-mwbjc" Jul 23 15:12:56.733196 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.733161 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/15c40f6c-249b-4f48-a8ae-f8b6da077e0e-hosts-file\") pod \"node-resolver-mwbjc\" (UID: \"15c40f6c-249b-4f48-a8ae-f8b6da077e0e\") " pod="openshift-dns/node-resolver-mwbjc" Jul 23 15:12:56.733368 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.733282 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/15c40f6c-249b-4f48-a8ae-f8b6da077e0e-hosts-file\") pod \"node-resolver-mwbjc\" (UID: \"15c40f6c-249b-4f48-a8ae-f8b6da077e0e\") " pod="openshift-dns/node-resolver-mwbjc" Jul 23 15:12:56.733440 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.733392 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15c40f6c-249b-4f48-a8ae-f8b6da077e0e-tmp-dir\") pod \"node-resolver-mwbjc\" (UID: \"15c40f6c-249b-4f48-a8ae-f8b6da077e0e\") " pod="openshift-dns/node-resolver-mwbjc" Jul 23 15:12:56.733777 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.733513 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg72s\" (UniqueName: \"kubernetes.io/projected/15c40f6c-249b-4f48-a8ae-f8b6da077e0e-kube-api-access-kg72s\") pod \"node-resolver-mwbjc\" (UID: \"15c40f6c-249b-4f48-a8ae-f8b6da077e0e\") " pod="openshift-dns/node-resolver-mwbjc" Jul 23 15:12:56.733899 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.733849 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15c40f6c-249b-4f48-a8ae-f8b6da077e0e-tmp-dir\") pod \"node-resolver-mwbjc\" (UID: \"15c40f6c-249b-4f48-a8ae-f8b6da077e0e\") " pod="openshift-dns/node-resolver-mwbjc" Jul 23 15:12:56.743844 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.743818 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg72s\" (UniqueName: \"kubernetes.io/projected/15c40f6c-249b-4f48-a8ae-f8b6da077e0e-kube-api-access-kg72s\") pod \"node-resolver-mwbjc\" (UID: \"15c40f6c-249b-4f48-a8ae-f8b6da077e0e\") " pod="openshift-dns/node-resolver-mwbjc" Jul 23 15:12:56.818032 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:56.817948 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mwbjc" Jul 23 15:12:56.840380 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:12:56.840348 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c40f6c_249b_4f48_a8ae_f8b6da077e0e.slice/crio-f1f047458256c47b3284f40d2400dc6de467025062b6794df6139e5c368864e0 WatchSource:0}: Error finding container f1f047458256c47b3284f40d2400dc6de467025062b6794df6139e5c368864e0: Status 404 returned error can't find the container with id f1f047458256c47b3284f40d2400dc6de467025062b6794df6139e5c368864e0 Jul 23 15:12:57.052729 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:57.052691 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ltmrf"] Jul 23 15:12:57.052901 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:57.052865 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:57.052989 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:57.052964 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:57.055238 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:57.055201 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wl9qs"] Jul 23 15:12:57.055366 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:57.055261 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n5624"] Jul 23 15:12:57.055366 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:57.055366 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:57.055486 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:57.055463 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:57.055547 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:57.055532 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:57.055616 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:57.055598 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:57.576556 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:57.576518 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mwbjc" event={"ID":"15c40f6c-249b-4f48-a8ae-f8b6da077e0e","Type":"ContainerStarted","Data":"772eafb09f8eb1fc7e00878151f640783be874a65f82c665404b343e083926ce"} Jul 23 15:12:57.577074 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:57.576577 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mwbjc" event={"ID":"15c40f6c-249b-4f48-a8ae-f8b6da077e0e","Type":"ContainerStarted","Data":"f1f047458256c47b3284f40d2400dc6de467025062b6794df6139e5c368864e0"} Jul 23 15:12:57.578747 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:57.578717 2570 generic.go:356] "Generic (PLEG): container finished" podID="8028e717-3208-45d4-a49f-117180037b85" containerID="46e1acceef971c80f2e6b969b1619cdd34b06973a781ad58b88f69bd3712f051" exitCode=0 Jul 23 15:12:57.578853 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:57.578797 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bhpc" event={"ID":"8028e717-3208-45d4-a49f-117180037b85","Type":"ContainerDied","Data":"46e1acceef971c80f2e6b969b1619cdd34b06973a781ad58b88f69bd3712f051"} Jul 23 15:12:57.595056 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:57.595004 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mwbjc" podStartSLOduration=1.594987394 podStartE2EDuration="1.594987394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:12:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:12:57.594894093 +0000 UTC m=+31.210941357" watchObservedRunningTime="2026-07-23 15:12:57.594987394 +0000 UTC m=+31.211034657" Jul 23 15:12:58.434854 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:58.434475 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:12:58.434854 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:58.434638 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:12:58.435792 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:58.435472 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:12:58.435792 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:58.435609 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:12:58.435792 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:58.435666 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:12:58.435792 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:12:58.435753 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:12:59.586418 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:59.586208 2570 generic.go:356] "Generic (PLEG): container finished" podID="8028e717-3208-45d4-a49f-117180037b85" containerID="7b1ad2f051e0eade2ca1d317f89aba4cc05b76cbd03512989b3e7de3a5f7428b" exitCode=0 Jul 23 15:12:59.586418 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:12:59.586288 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bhpc" event={"ID":"8028e717-3208-45d4-a49f-117180037b85","Type":"ContainerDied","Data":"7b1ad2f051e0eade2ca1d317f89aba4cc05b76cbd03512989b3e7de3a5f7428b"} Jul 23 15:13:00.432852 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:00.432819 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:13:00.433021 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:00.432819 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:13:00.433021 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:00.432950 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ltmrf" podUID="5354f3d7-6d0a-48bf-b8c1-85949c45d9d2" Jul 23 15:13:00.433021 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:00.433003 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n5624" podUID="7b189e73-6f40-467e-9bf1-5505c5a7194c" Jul 23 15:13:00.433021 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:00.432836 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:13:00.433250 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:00.433202 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wl9qs" podUID="7cff11e5-8f10-4219-9ed9-76a3c70061c8" Jul 23 15:13:01.779935 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:01.779892 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:13:01.780551 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:01.779951 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs\") pod \"network-metrics-daemon-n5624\" (UID: \"7b189e73-6f40-467e-9bf1-5505c5a7194c\") " pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:13:01.780551 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:01.780035 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 15:13:01.780551 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:01.780081 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:13:01.780551 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:01.780144 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs podName:7b189e73-6f40-467e-9bf1-5505c5a7194c nodeName:}" failed. No retries permitted until 2026-07-23 15:13:33.780120196 +0000 UTC m=+67.396167438 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs") pod "network-metrics-daemon-n5624" (UID: "7b189e73-6f40-467e-9bf1-5505c5a7194c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:13:01.780551 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:01.780161 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret podName:7cff11e5-8f10-4219-9ed9-76a3c70061c8 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:33.780152231 +0000 UTC m=+67.396199474 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret") pod "global-pull-secret-syncer-wl9qs" (UID: "7cff11e5-8f10-4219-9ed9-76a3c70061c8") : object "kube-system"/"original-pull-secret" not registered Jul 23 15:13:01.880526 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:01.880494 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrxqw\" (UniqueName: \"kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw\") pod \"network-check-target-ltmrf\" (UID: \"5354f3d7-6d0a-48bf-b8c1-85949c45d9d2\") " pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:13:01.880729 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:01.880709 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 15:13:01.880780 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:01.880734 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 15:13:01.880780 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:01.880745 2570 projected.go:196] Error preparing data for projected volume kube-api-access-zrxqw for pod openshift-network-diagnostics/network-check-target-ltmrf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:13:01.880854 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:01.880815 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw podName:5354f3d7-6d0a-48bf-b8c1-85949c45d9d2 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:33.880794245 +0000 UTC m=+67.496841490 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zrxqw" (UniqueName: "kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw") pod "network-check-target-ltmrf" (UID: "5354f3d7-6d0a-48bf-b8c1-85949c45d9d2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:13:02.214039 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.214006 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-63.ec2.internal" event="NodeReady" Jul 23 15:13:02.214209 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.214168 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 15:13:02.272533 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.272497 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8"] Jul 23 15:13:02.275186 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.275156 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt"] Jul 23 15:13:02.275464 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.275438 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:02.276992 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.276971 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-jhn9s"] Jul 23 15:13:02.277098 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.277066 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" Jul 23 15:13:02.278068 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.277883 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 23 15:13:02.279003 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.278987 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-758d7fd447-qkrz6"] Jul 23 15:13:02.279120 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.279101 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.281003 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.280983 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-pnzkz"] Jul 23 15:13:02.281106 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.281058 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.282833 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.282812 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-s6czh"] Jul 23 15:13:02.283404 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.283005 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-pnzkz" Jul 23 15:13:02.283404 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.283120 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 23 15:13:02.283404 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.283147 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 15:13:02.283404 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.283164 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 15:13:02.283614 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.283528 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:13:02.283662 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.283646 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kqqgz\"" Jul 23 15:13:02.284455 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.284309 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 23 15:13:02.284455 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.284324 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 23 15:13:02.284455 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.284334 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 23 15:13:02.284455 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.284378 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-sbbsb\"" Jul 23 15:13:02.284926 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.284611 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-74d5f4575d-4d249"] Jul 23 15:13:02.284926 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.284740 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:02.285943 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.285927 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 23 15:13:02.287007 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.286286 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q"] Jul 23 15:13:02.287007 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.286384 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.288081 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.288063 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg"] Jul 23 15:13:02.288201 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.288184 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" Jul 23 15:13:02.289885 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.289864 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" Jul 23 15:13:02.303239 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.303178 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 23 15:13:02.303909 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.303845 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mfk6w\"" Jul 23 15:13:02.304411 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.304390 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-5bzsz\"" Jul 23 15:13:02.304939 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.304918 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 15:13:02.305150 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.305135 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 15:13:02.305372 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.305352 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:13:02.305503 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.305209 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 15:13:02.305567 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.305154 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 23 15:13:02.306126 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.306090 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 23 15:13:02.306954 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.306934 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 23 15:13:02.307257 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.307238 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 15:13:02.307410 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.307395 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 23 15:13:02.307505 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.307064 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 23 15:13:02.307585 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.307528 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:13:02.307666 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.307582 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wqjcr\"" Jul 23 15:13:02.307666 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.307131 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 23 15:13:02.307666 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.307665 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zt8sw\"" Jul 23 15:13:02.307953 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.307176 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:13:02.307953 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.307827 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:13:02.307953 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.307852 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-l6cfn\"" Jul 23 15:13:02.307953 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.307934 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 15:13:02.308188 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.307941 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 15:13:02.308188 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.308081 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 23 15:13:02.308284 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.308198 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 15:13:02.308418 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.308368 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 23 15:13:02.308570 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.307045 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-phbzg\"" Jul 23 15:13:02.308790 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.308760 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 23 15:13:02.308886 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.308800 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 23 15:13:02.308966 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.308898 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 23 15:13:02.309141 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.309110 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 23 15:13:02.309270 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.309253 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 23 15:13:02.309406 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.309178 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 15:13:02.310059 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.310041 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 23 15:13:02.310361 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.310343 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-pnzkz"] Jul 23 15:13:02.310585 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.310566 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-khmmr\"" Jul 23 15:13:02.310763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.310732 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 23 15:13:02.311240 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.311195 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt"] Jul 23 15:13:02.312539 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.312344 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-jhn9s"] Jul 23 15:13:02.312964 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.312942 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-s6czh"] Jul 23 15:13:02.313865 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.313839 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-758d7fd447-qkrz6"] Jul 23 15:13:02.314994 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.314945 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74d5f4575d-4d249"] Jul 23 15:13:02.316005 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.315861 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8"] Jul 23 15:13:02.316662 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.316644 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg"] Jul 23 15:13:02.328884 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.328860 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-tnxg9"] Jul 23 15:13:02.331936 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.331904 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x"] Jul 23 15:13:02.332175 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.332138 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tnxg9" Jul 23 15:13:02.334055 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.334037 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl"] Jul 23 15:13:02.334179 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.334160 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" Jul 23 15:13:02.335542 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.335522 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 15:13:02.335627 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.335575 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 15:13:02.335891 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.335875 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-g567n\"" Jul 23 15:13:02.336064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.336043 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8sl9b"] Jul 23 15:13:02.336358 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.336338 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.336888 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.336869 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 23 15:13:02.337608 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.337588 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 23 15:13:02.338064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.338044 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 23 15:13:02.338418 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.338400 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7"] Jul 23 15:13:02.338534 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.338517 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8sl9b" Jul 23 15:13:02.340508 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.340487 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-whcfg"] Jul 23 15:13:02.340644 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.340625 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7" Jul 23 15:13:02.342581 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.342562 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q"] Jul 23 15:13:02.342690 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.342674 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:02.342880 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.342862 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 23 15:13:02.343622 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.343602 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 23 15:13:02.343758 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.343737 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 15:13:02.343758 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.343666 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 15:13:02.343758 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.343758 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 15:13:02.343944 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.343853 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 23 15:13:02.343944 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.343932 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xt5k8\"" Jul 23 15:13:02.344017 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.343955 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 23 15:13:02.350448 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.350429 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 15:13:02.353162 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.353145 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 15:13:02.353331 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.353238 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 23 15:13:02.353331 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.353240 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 23 15:13:02.353555 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.353538 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-b24z4\"" Jul 23 15:13:02.353985 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.353970 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7fsth\"" Jul 23 15:13:02.368339 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.367696 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x"] Jul 23 15:13:02.368970 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.368949 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7"] Jul 23 15:13:02.370111 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.370087 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-tnxg9"] Jul 23 15:13:02.371336 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.371317 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8sl9b"] Jul 23 15:13:02.372825 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.372806 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl"] Jul 23 15:13:02.380505 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.380483 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-whcfg"] Jul 23 15:13:02.384408 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.384381 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71d3e351-b9cf-47fb-89b7-d69324c7a146-serving-cert\") pod \"service-ca-operator-5bcbc94c78-2l9gg\" (UID: \"71d3e351-b9cf-47fb-89b7-d69324c7a146\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" Jul 23 15:13:02.384569 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.384546 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m22c\" (UniqueName: \"kubernetes.io/projected/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-kube-api-access-7m22c\") pod \"cluster-samples-operator-79b7c55f87-v9g2q\" (UID: \"1b97d33c-4f23-4761-8dbe-7579d7b61a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" Jul 23 15:13:02.384683 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.384667 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/48561ae8-8b4f-4367-a2ef-20584d90b8e4-snapshots\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.384775 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.384761 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48561ae8-8b4f-4367-a2ef-20584d90b8e4-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.384876 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.384863 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2vsr\" (UniqueName: \"kubernetes.io/projected/71d3e351-b9cf-47fb-89b7-d69324c7a146-kube-api-access-p2vsr\") pod \"service-ca-operator-5bcbc94c78-2l9gg\" (UID: \"71d3e351-b9cf-47fb-89b7-d69324c7a146\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" Jul 23 15:13:02.384980 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.384962 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klgpz\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-kube-api-access-klgpz\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.385064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.384994 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f9b322c9-ea12-44a0-994a-b481ef39e4e3-image-registry-private-configuration\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.385064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385026 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:02.385064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385049 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-certificates\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.385172 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385121 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/48561ae8-8b4f-4367-a2ef-20584d90b8e4-serving-cert\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.385204 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385176 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/36dc10ed-f042-4994-8241-113f098485d6-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-w5gjt\" (UID: \"36dc10ed-f042-4994-8241-113f098485d6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" Jul 23 15:13:02.385252 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385235 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f9b322c9-ea12-44a0-994a-b481ef39e4e3-installation-pull-secrets\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.385311 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385265 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-bound-sa-token\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.385377 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385289 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcc78\" (UniqueName: \"kubernetes.io/projected/c59394de-e921-4c8e-ab5c-d310242a8149-kube-api-access-bcc78\") pod \"volume-data-source-validator-78cf4c6f6-pnzkz\" (UID: \"c59394de-e921-4c8e-ab5c-d310242a8149\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-pnzkz" Jul 23 15:13:02.385429 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385394 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71d3e351-b9cf-47fb-89b7-d69324c7a146-config\") pod \"service-ca-operator-5bcbc94c78-2l9gg\" (UID: \"71d3e351-b9cf-47fb-89b7-d69324c7a146\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" Jul 23 15:13:02.385479 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385428 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f9b322c9-ea12-44a0-994a-b481ef39e4e3-trusted-ca\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.385479 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385456 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fsqt\" (UniqueName: \"kubernetes.io/projected/5522141f-9f92-48ad-a954-7c0bff719ebd-kube-api-access-8fsqt\") pod \"console-operator-565b7b569d-s6czh\" (UID: \"5522141f-9f92-48ad-a954-7c0bff719ebd\") " pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:02.385574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385487 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/45875b2d-d207-49c6-8b86-aa50e0198929-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:02.385574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385515 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48561ae8-8b4f-4367-a2ef-20584d90b8e4-tmp\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.385574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385544 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmsb5\" (UniqueName: \"kubernetes.io/projected/f97ae7a3-b422-44a9-8942-752524470865-kube-api-access-qmsb5\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.385700 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385573 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5522141f-9f92-48ad-a954-7c0bff719ebd-serving-cert\") pod \"console-operator-565b7b569d-s6czh\" (UID: \"5522141f-9f92-48ad-a954-7c0bff719ebd\") " pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:02.385700 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385598 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-stats-auth\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.385700 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385627 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ddwz\" (UniqueName: \"kubernetes.io/projected/36dc10ed-f042-4994-8241-113f098485d6-kube-api-access-7ddwz\") pod \"kube-storage-version-migrator-operator-7658d9b749-w5gjt\" (UID: \"36dc10ed-f042-4994-8241-113f098485d6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" Jul 23 15:13:02.385700 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385664 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.385700 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385691 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l5f6\" (UniqueName: \"kubernetes.io/projected/48561ae8-8b4f-4367-a2ef-20584d90b8e4-kube-api-access-7l5f6\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.385924 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385708 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-v9g2q\" (UID: \"1b97d33c-4f23-4761-8dbe-7579d7b61a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" Jul 23 15:13:02.385924 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385740 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.385924 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385797 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz4hj\" (UniqueName: \"kubernetes.io/projected/45875b2d-d207-49c6-8b86-aa50e0198929-kube-api-access-qz4hj\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:02.385924 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385843 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48561ae8-8b4f-4367-a2ef-20584d90b8e4-service-ca-bundle\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.385924 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385873 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.385924 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385891 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5522141f-9f92-48ad-a954-7c0bff719ebd-config\") pod \"console-operator-565b7b569d-s6czh\" (UID: \"5522141f-9f92-48ad-a954-7c0bff719ebd\") " pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:02.386201 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.385961 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5522141f-9f92-48ad-a954-7c0bff719ebd-trusted-ca\") pod \"console-operator-565b7b569d-s6czh\" (UID: \"5522141f-9f92-48ad-a954-7c0bff719ebd\") " pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:02.386201 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.386019 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-default-certificate\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.386201 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.386074 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f9b322c9-ea12-44a0-994a-b481ef39e4e3-ca-trust-extracted\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.386201 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.386105 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/36dc10ed-f042-4994-8241-113f098485d6-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-w5gjt\" (UID: \"36dc10ed-f042-4994-8241-113f098485d6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" Jul 23 15:13:02.432855 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.432827 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:13:02.432855 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.432830 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:13:02.433062 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.432992 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:13:02.435708 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.435684 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 15:13:02.436029 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.436008 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7hjlh\"" Jul 23 15:13:02.436029 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.436017 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 15:13:02.436441 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.436420 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-srpb5\"" Jul 23 15:13:02.487439 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487351 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/48561ae8-8b4f-4367-a2ef-20584d90b8e4-serving-cert\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.487439 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487396 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/36dc10ed-f042-4994-8241-113f098485d6-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-w5gjt\" (UID: \"36dc10ed-f042-4994-8241-113f098485d6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" Jul 23 15:13:02.487439 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487427 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fee87863-6596-4de1-bf68-002c95fb789a-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f95d4c776-cj58x\" (UID: \"fee87863-6596-4de1-bf68-002c95fb789a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" Jul 23 15:13:02.487705 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487459 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f9b322c9-ea12-44a0-994a-b481ef39e4e3-installation-pull-secrets\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.487705 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487484 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-bound-sa-token\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.487705 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487508 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcc78\" (UniqueName: \"kubernetes.io/projected/c59394de-e921-4c8e-ab5c-d310242a8149-kube-api-access-bcc78\") pod \"volume-data-source-validator-78cf4c6f6-pnzkz\" (UID: \"c59394de-e921-4c8e-ab5c-d310242a8149\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-pnzkz" Jul 23 15:13:02.487705 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487536 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71d3e351-b9cf-47fb-89b7-d69324c7a146-config\") pod \"service-ca-operator-5bcbc94c78-2l9gg\" (UID: \"71d3e351-b9cf-47fb-89b7-d69324c7a146\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" Jul 23 15:13:02.487705 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487561 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f9b322c9-ea12-44a0-994a-b481ef39e4e3-trusted-ca\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.487705 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487584 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnfxj\" (UniqueName: \"kubernetes.io/projected/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-kube-api-access-bnfxj\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.487705 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487621 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d7ff\" (UniqueName: \"kubernetes.io/projected/e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9-kube-api-access-6d7ff\") pod \"ingress-canary-8sl9b\" (UID: \"e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9\") " pod="openshift-ingress-canary/ingress-canary-8sl9b" Jul 23 15:13:02.487705 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487649 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fsqt\" (UniqueName: \"kubernetes.io/projected/5522141f-9f92-48ad-a954-7c0bff719ebd-kube-api-access-8fsqt\") pod \"console-operator-565b7b569d-s6czh\" (UID: \"5522141f-9f92-48ad-a954-7c0bff719ebd\") " pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:02.487705 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487675 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvq4j\" (UniqueName: \"kubernetes.io/projected/78c82b4f-1367-497e-8767-7372b0c2d9e4-kube-api-access-jvq4j\") pod \"managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7\" (UID: \"78c82b4f-1367-497e-8767-7372b0c2d9e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7" Jul 23 15:13:02.488064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487708 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/45875b2d-d207-49c6-8b86-aa50e0198929-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:02.488064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487734 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48561ae8-8b4f-4367-a2ef-20584d90b8e4-tmp\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.488064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487757 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvhvp\" (UniqueName: \"kubernetes.io/projected/38a36f07-0e32-46d6-9a99-2bd74ed87244-kube-api-access-gvhvp\") pod \"network-check-source-7bc5b6bf85-tnxg9\" (UID: \"38a36f07-0e32-46d6-9a99-2bd74ed87244\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tnxg9" Jul 23 15:13:02.488064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487779 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmsb5\" (UniqueName: \"kubernetes.io/projected/f97ae7a3-b422-44a9-8942-752524470865-kube-api-access-qmsb5\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.488064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487800 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-tmp-dir\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:02.488064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487823 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5522141f-9f92-48ad-a954-7c0bff719ebd-serving-cert\") pod \"console-operator-565b7b569d-s6czh\" (UID: \"5522141f-9f92-48ad-a954-7c0bff719ebd\") " pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:02.488064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487846 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-stats-auth\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.488064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487878 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.488064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487904 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:02.488064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487934 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ddwz\" (UniqueName: \"kubernetes.io/projected/36dc10ed-f042-4994-8241-113f098485d6-kube-api-access-7ddwz\") pod \"kube-storage-version-migrator-operator-7658d9b749-w5gjt\" (UID: \"36dc10ed-f042-4994-8241-113f098485d6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" Jul 23 15:13:02.488064 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.487982 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.488520 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.488188 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-ca\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.488520 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.488251 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 15:13:02.488520 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.488283 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l5f6\" (UniqueName: \"kubernetes.io/projected/48561ae8-8b4f-4367-a2ef-20584d90b8e4-kube-api-access-7l5f6\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.488520 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.488343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-v9g2q\" (UID: \"1b97d33c-4f23-4761-8dbe-7579d7b61a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" Jul 23 15:13:02.488520 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.488374 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs podName:f97ae7a3-b422-44a9-8942-752524470865 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:02.988353811 +0000 UTC m=+36.604401056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs") pod "router-default-74d5f4575d-4d249" (UID: "f97ae7a3-b422-44a9-8942-752524470865") : secret "router-metrics-certs-default" not found Jul 23 15:13:02.488520 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.488438 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 15:13:02.488520 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.488483 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls podName:1b97d33c-4f23-4761-8dbe-7579d7b61a5e nodeName:}" failed. No retries permitted until 2026-07-23 15:13:02.988473885 +0000 UTC m=+36.604521127 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-v9g2q" (UID: "1b97d33c-4f23-4761-8dbe-7579d7b61a5e") : secret "samples-operator-tls" not found Jul 23 15:13:02.488520 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.488508 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.488911 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.488855 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 15:13:02.488911 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.488872 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758d7fd447-qkrz6: secret "image-registry-tls" not found Jul 23 15:13:02.488911 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.488877 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71d3e351-b9cf-47fb-89b7-d69324c7a146-config\") pod \"service-ca-operator-5bcbc94c78-2l9gg\" (UID: \"71d3e351-b9cf-47fb-89b7-d69324c7a146\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" Jul 23 15:13:02.489071 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.488936 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls podName:f9b322c9-ea12-44a0-994a-b481ef39e4e3 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:02.988920388 +0000 UTC m=+36.604967646 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls") pod "image-registry-758d7fd447-qkrz6" (UID: "f9b322c9-ea12-44a0-994a-b481ef39e4e3") : secret "image-registry-tls" not found Jul 23 15:13:02.489071 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.488992 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz4hj\" (UniqueName: \"kubernetes.io/projected/45875b2d-d207-49c6-8b86-aa50e0198929-kube-api-access-qz4hj\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:02.489071 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489059 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/45875b2d-d207-49c6-8b86-aa50e0198929-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:02.489216 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489082 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48561ae8-8b4f-4367-a2ef-20584d90b8e4-service-ca-bundle\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.489216 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489139 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-config-volume\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:02.489382 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489361 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-additional-service-ca\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.489382 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489368 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48561ae8-8b4f-4367-a2ef-20584d90b8e4-tmp\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.489503 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489456 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.489556 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.489542 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle podName:f97ae7a3-b422-44a9-8942-752524470865 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:02.989528612 +0000 UTC m=+36.605575872 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle") pod "router-default-74d5f4575d-4d249" (UID: "f97ae7a3-b422-44a9-8942-752524470865") : configmap references non-existent config key: service-ca.crt Jul 23 15:13:02.489615 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489578 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5522141f-9f92-48ad-a954-7c0bff719ebd-config\") pod \"console-operator-565b7b569d-s6czh\" (UID: \"5522141f-9f92-48ad-a954-7c0bff719ebd\") " pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:02.489615 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489603 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5522141f-9f92-48ad-a954-7c0bff719ebd-trusted-ca\") pod \"console-operator-565b7b569d-s6czh\" (UID: \"5522141f-9f92-48ad-a954-7c0bff719ebd\") " pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:02.489714 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489624 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-default-certificate\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.489714 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489647 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-hub\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.489714 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489663 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48561ae8-8b4f-4367-a2ef-20584d90b8e4-service-ca-bundle\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.489714 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489674 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f9b322c9-ea12-44a0-994a-b481ef39e4e3-ca-trust-extracted\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.489714 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489695 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9-cert\") pod \"ingress-canary-8sl9b\" (UID: \"e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9\") " pod="openshift-ingress-canary/ingress-canary-8sl9b" Jul 23 15:13:02.489955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489721 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/36dc10ed-f042-4994-8241-113f098485d6-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-w5gjt\" (UID: \"36dc10ed-f042-4994-8241-113f098485d6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" Jul 23 15:13:02.489955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489744 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfb6z\" (UniqueName: \"kubernetes.io/projected/fee87863-6596-4de1-bf68-002c95fb789a-kube-api-access-kfb6z\") pod \"klusterlet-addon-workmgr-5f95d4c776-cj58x\" (UID: \"fee87863-6596-4de1-bf68-002c95fb789a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" Jul 23 15:13:02.489955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489768 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.489955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489797 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71d3e351-b9cf-47fb-89b7-d69324c7a146-serving-cert\") pod \"service-ca-operator-5bcbc94c78-2l9gg\" (UID: \"71d3e351-b9cf-47fb-89b7-d69324c7a146\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" Jul 23 15:13:02.489955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489822 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m22c\" (UniqueName: \"kubernetes.io/projected/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-kube-api-access-7m22c\") pod \"cluster-samples-operator-79b7c55f87-v9g2q\" (UID: \"1b97d33c-4f23-4761-8dbe-7579d7b61a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" Jul 23 15:13:02.489955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489842 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krqqr\" (UniqueName: \"kubernetes.io/projected/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-kube-api-access-krqqr\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:02.489955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489868 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/48561ae8-8b4f-4367-a2ef-20584d90b8e4-snapshots\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.489955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489889 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48561ae8-8b4f-4367-a2ef-20584d90b8e4-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.489955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489912 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2vsr\" (UniqueName: \"kubernetes.io/projected/71d3e351-b9cf-47fb-89b7-d69324c7a146-kube-api-access-p2vsr\") pod \"service-ca-operator-5bcbc94c78-2l9gg\" (UID: \"71d3e351-b9cf-47fb-89b7-d69324c7a146\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" Jul 23 15:13:02.489955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489925 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f9b322c9-ea12-44a0-994a-b481ef39e4e3-trusted-ca\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.489955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489936 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klgpz\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-kube-api-access-klgpz\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.490669 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489962 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f9b322c9-ea12-44a0-994a-b481ef39e4e3-image-registry-private-configuration\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.490669 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.489982 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fee87863-6596-4de1-bf68-002c95fb789a-tmp\") pod \"klusterlet-addon-workmgr-5f95d4c776-cj58x\" (UID: \"fee87863-6596-4de1-bf68-002c95fb789a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" Jul 23 15:13:02.490669 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.490013 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78c82b4f-1367-497e-8767-7372b0c2d9e4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7\" (UID: \"78c82b4f-1367-497e-8767-7372b0c2d9e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7" Jul 23 15:13:02.490669 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.490042 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:02.490669 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.490065 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-certificates\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.490669 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.490615 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-certificates\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.491379 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.491354 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/48561ae8-8b4f-4367-a2ef-20584d90b8e4-snapshots\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.492414 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.492130 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48561ae8-8b4f-4367-a2ef-20584d90b8e4-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.492414 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.492354 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-stats-auth\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.492414 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.492376 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f9b322c9-ea12-44a0-994a-b481ef39e4e3-installation-pull-secrets\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.492414 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.492404 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5522141f-9f92-48ad-a954-7c0bff719ebd-config\") pod \"console-operator-565b7b569d-s6czh\" (UID: \"5522141f-9f92-48ad-a954-7c0bff719ebd\") " pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:02.492658 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.492475 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-default-certificate\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.492658 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.492613 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 15:13:02.492761 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.492661 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls podName:45875b2d-d207-49c6-8b86-aa50e0198929 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:02.992646129 +0000 UTC m=+36.608693378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9mmv8" (UID: "45875b2d-d207-49c6-8b86-aa50e0198929") : secret "cluster-monitoring-operator-tls" not found Jul 23 15:13:02.492761 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.492723 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f9b322c9-ea12-44a0-994a-b481ef39e4e3-ca-trust-extracted\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.492876 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.492790 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5522141f-9f92-48ad-a954-7c0bff719ebd-serving-cert\") pod \"console-operator-565b7b569d-s6czh\" (UID: \"5522141f-9f92-48ad-a954-7c0bff719ebd\") " pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:02.493005 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.492981 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71d3e351-b9cf-47fb-89b7-d69324c7a146-serving-cert\") pod \"service-ca-operator-5bcbc94c78-2l9gg\" (UID: \"71d3e351-b9cf-47fb-89b7-d69324c7a146\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" Jul 23 15:13:02.493345 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.493317 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5522141f-9f92-48ad-a954-7c0bff719ebd-trusted-ca\") pod \"console-operator-565b7b569d-s6czh\" (UID: \"5522141f-9f92-48ad-a954-7c0bff719ebd\") " pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:02.493923 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.493849 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/36dc10ed-f042-4994-8241-113f098485d6-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-w5gjt\" (UID: \"36dc10ed-f042-4994-8241-113f098485d6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" Jul 23 15:13:02.494258 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.494209 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/36dc10ed-f042-4994-8241-113f098485d6-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-w5gjt\" (UID: \"36dc10ed-f042-4994-8241-113f098485d6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" Jul 23 15:13:02.494666 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.494646 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/48561ae8-8b4f-4367-a2ef-20584d90b8e4-serving-cert\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.494927 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.494902 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f9b322c9-ea12-44a0-994a-b481ef39e4e3-image-registry-private-configuration\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.517018 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.516985 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-bound-sa-token\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.517268 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.517251 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fsqt\" (UniqueName: \"kubernetes.io/projected/5522141f-9f92-48ad-a954-7c0bff719ebd-kube-api-access-8fsqt\") pod \"console-operator-565b7b569d-s6czh\" (UID: \"5522141f-9f92-48ad-a954-7c0bff719ebd\") " pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:02.523997 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.523942 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m22c\" (UniqueName: \"kubernetes.io/projected/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-kube-api-access-7m22c\") pod \"cluster-samples-operator-79b7c55f87-v9g2q\" (UID: \"1b97d33c-4f23-4761-8dbe-7579d7b61a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" Jul 23 15:13:02.526535 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.526507 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcc78\" (UniqueName: \"kubernetes.io/projected/c59394de-e921-4c8e-ab5c-d310242a8149-kube-api-access-bcc78\") pod \"volume-data-source-validator-78cf4c6f6-pnzkz\" (UID: \"c59394de-e921-4c8e-ab5c-d310242a8149\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-pnzkz" Jul 23 15:13:02.527213 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.527099 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmsb5\" (UniqueName: \"kubernetes.io/projected/f97ae7a3-b422-44a9-8942-752524470865-kube-api-access-qmsb5\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.528146 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.528092 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l5f6\" (UniqueName: \"kubernetes.io/projected/48561ae8-8b4f-4367-a2ef-20584d90b8e4-kube-api-access-7l5f6\") pod \"insights-operator-555d9d7885-jhn9s\" (UID: \"48561ae8-8b4f-4367-a2ef-20584d90b8e4\") " pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.528968 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.528926 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ddwz\" (UniqueName: \"kubernetes.io/projected/36dc10ed-f042-4994-8241-113f098485d6-kube-api-access-7ddwz\") pod \"kube-storage-version-migrator-operator-7658d9b749-w5gjt\" (UID: \"36dc10ed-f042-4994-8241-113f098485d6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" Jul 23 15:13:02.529242 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.529220 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klgpz\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-kube-api-access-klgpz\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.531466 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.531444 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz4hj\" (UniqueName: \"kubernetes.io/projected/45875b2d-d207-49c6-8b86-aa50e0198929-kube-api-access-qz4hj\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:02.531725 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.531701 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2vsr\" (UniqueName: \"kubernetes.io/projected/71d3e351-b9cf-47fb-89b7-d69324c7a146-kube-api-access-p2vsr\") pod \"service-ca-operator-5bcbc94c78-2l9gg\" (UID: \"71d3e351-b9cf-47fb-89b7-d69324c7a146\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" Jul 23 15:13:02.590792 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.590646 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-config-volume\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:02.590792 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.590698 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-additional-service-ca\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.590792 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.590762 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-hub\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.590792 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.590795 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9-cert\") pod \"ingress-canary-8sl9b\" (UID: \"e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9\") " pod="openshift-ingress-canary/ingress-canary-8sl9b" Jul 23 15:13:02.591148 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.590829 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfb6z\" (UniqueName: \"kubernetes.io/projected/fee87863-6596-4de1-bf68-002c95fb789a-kube-api-access-kfb6z\") pod \"klusterlet-addon-workmgr-5f95d4c776-cj58x\" (UID: \"fee87863-6596-4de1-bf68-002c95fb789a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" Jul 23 15:13:02.591148 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.590855 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.591148 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.590883 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krqqr\" (UniqueName: \"kubernetes.io/projected/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-kube-api-access-krqqr\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:02.591148 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.590916 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fee87863-6596-4de1-bf68-002c95fb789a-tmp\") pod \"klusterlet-addon-workmgr-5f95d4c776-cj58x\" (UID: \"fee87863-6596-4de1-bf68-002c95fb789a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" Jul 23 15:13:02.591148 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.590947 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78c82b4f-1367-497e-8767-7372b0c2d9e4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7\" (UID: \"78c82b4f-1367-497e-8767-7372b0c2d9e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7" Jul 23 15:13:02.591148 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.591000 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fee87863-6596-4de1-bf68-002c95fb789a-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f95d4c776-cj58x\" (UID: \"fee87863-6596-4de1-bf68-002c95fb789a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" Jul 23 15:13:02.591148 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.591031 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnfxj\" (UniqueName: \"kubernetes.io/projected/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-kube-api-access-bnfxj\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.591148 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.591052 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d7ff\" (UniqueName: \"kubernetes.io/projected/e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9-kube-api-access-6d7ff\") pod \"ingress-canary-8sl9b\" (UID: \"e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9\") " pod="openshift-ingress-canary/ingress-canary-8sl9b" Jul 23 15:13:02.591148 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.591073 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvq4j\" (UniqueName: \"kubernetes.io/projected/78c82b4f-1367-497e-8767-7372b0c2d9e4-kube-api-access-jvq4j\") pod \"managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7\" (UID: \"78c82b4f-1367-497e-8767-7372b0c2d9e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7" Jul 23 15:13:02.592041 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.591941 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvhvp\" (UniqueName: \"kubernetes.io/projected/38a36f07-0e32-46d6-9a99-2bd74ed87244-kube-api-access-gvhvp\") pod \"network-check-source-7bc5b6bf85-tnxg9\" (UID: \"38a36f07-0e32-46d6-9a99-2bd74ed87244\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tnxg9" Jul 23 15:13:02.592041 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.592007 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-tmp-dir\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:02.592193 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.592060 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.592193 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.592089 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:02.592193 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.592135 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-ca\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.592944 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.592392 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-tmp-dir\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:02.592944 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.591950 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fee87863-6596-4de1-bf68-002c95fb789a-tmp\") pod \"klusterlet-addon-workmgr-5f95d4c776-cj58x\" (UID: \"fee87863-6596-4de1-bf68-002c95fb789a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" Jul 23 15:13:02.593093 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.592950 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-config-volume\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:02.593255 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.593228 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-additional-service-ca\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.593626 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.593352 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 15:13:02.593626 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.593433 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls podName:5e6d6e02-f435-4ccd-9b43-5c4f79d86b50 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:03.09341606 +0000 UTC m=+36.709463314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls") pod "dns-default-whcfg" (UID: "5e6d6e02-f435-4ccd-9b43-5c4f79d86b50") : secret "dns-default-metrics-tls" not found Jul 23 15:13:02.594869 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.594821 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.595031 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.595008 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9-cert\") pod \"ingress-canary-8sl9b\" (UID: \"e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9\") " pod="openshift-ingress-canary/ingress-canary-8sl9b" Jul 23 15:13:02.595137 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.595110 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-hub\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.595204 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.595132 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fee87863-6596-4de1-bf68-002c95fb789a-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f95d4c776-cj58x\" (UID: \"fee87863-6596-4de1-bf68-002c95fb789a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" Jul 23 15:13:02.595474 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.595453 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-ca\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.595680 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.595660 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.596548 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.596528 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/78c82b4f-1367-497e-8767-7372b0c2d9e4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7\" (UID: \"78c82b4f-1367-497e-8767-7372b0c2d9e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7" Jul 23 15:13:02.596762 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.596744 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" Jul 23 15:13:02.602851 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.602826 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvq4j\" (UniqueName: \"kubernetes.io/projected/78c82b4f-1367-497e-8767-7372b0c2d9e4-kube-api-access-jvq4j\") pod \"managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7\" (UID: \"78c82b4f-1367-497e-8767-7372b0c2d9e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7" Jul 23 15:13:02.605805 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.605782 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfb6z\" (UniqueName: \"kubernetes.io/projected/fee87863-6596-4de1-bf68-002c95fb789a-kube-api-access-kfb6z\") pod \"klusterlet-addon-workmgr-5f95d4c776-cj58x\" (UID: \"fee87863-6596-4de1-bf68-002c95fb789a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" Jul 23 15:13:02.607791 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.607738 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krqqr\" (UniqueName: \"kubernetes.io/projected/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-kube-api-access-krqqr\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:02.608955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.608927 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnfxj\" (UniqueName: \"kubernetes.io/projected/fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32-kube-api-access-bnfxj\") pod \"cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl\" (UID: \"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.609056 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.609041 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvhvp\" (UniqueName: \"kubernetes.io/projected/38a36f07-0e32-46d6-9a99-2bd74ed87244-kube-api-access-gvhvp\") pod \"network-check-source-7bc5b6bf85-tnxg9\" (UID: \"38a36f07-0e32-46d6-9a99-2bd74ed87244\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tnxg9" Jul 23 15:13:02.609708 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.609685 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d7ff\" (UniqueName: \"kubernetes.io/projected/e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9-kube-api-access-6d7ff\") pod \"ingress-canary-8sl9b\" (UID: \"e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9\") " pod="openshift-ingress-canary/ingress-canary-8sl9b" Jul 23 15:13:02.612498 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.612476 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-jhn9s" Jul 23 15:13:02.628132 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.628098 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-pnzkz" Jul 23 15:13:02.638384 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.638355 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:02.664500 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.664462 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" Jul 23 15:13:02.674706 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.674676 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tnxg9" Jul 23 15:13:02.682191 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.682168 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" Jul 23 15:13:02.697805 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.697778 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:13:02.707148 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.707120 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8sl9b" Jul 23 15:13:02.719918 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.719894 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7" Jul 23 15:13:02.996631 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.996590 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.997034 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.996745 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:02.997034 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.996783 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle podName:f97ae7a3-b422-44a9-8942-752524470865 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:03.996760195 +0000 UTC m=+37.612807445 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle") pod "router-default-74d5f4575d-4d249" (UID: "f97ae7a3-b422-44a9-8942-752524470865") : configmap references non-existent config key: service-ca.crt Jul 23 15:13:02.997034 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.996845 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 15:13:02.997034 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.996909 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls podName:45875b2d-d207-49c6-8b86-aa50e0198929 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:03.996894334 +0000 UTC m=+37.612941581 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9mmv8" (UID: "45875b2d-d207-49c6-8b86-aa50e0198929") : secret "cluster-monitoring-operator-tls" not found Jul 23 15:13:02.997034 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.996928 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:02.997034 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.996961 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-v9g2q\" (UID: \"1b97d33c-4f23-4761-8dbe-7579d7b61a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" Jul 23 15:13:02.997034 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:02.997028 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:02.997393 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.997052 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 15:13:02.997393 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.997089 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 15:13:02.997393 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.997101 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs podName:f97ae7a3-b422-44a9-8942-752524470865 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:03.997085093 +0000 UTC m=+37.613132346 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs") pod "router-default-74d5f4575d-4d249" (UID: "f97ae7a3-b422-44a9-8942-752524470865") : secret "router-metrics-certs-default" not found Jul 23 15:13:02.997393 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.997130 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls podName:1b97d33c-4f23-4761-8dbe-7579d7b61a5e nodeName:}" failed. No retries permitted until 2026-07-23 15:13:03.997118533 +0000 UTC m=+37.613165789 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-v9g2q" (UID: "1b97d33c-4f23-4761-8dbe-7579d7b61a5e") : secret "samples-operator-tls" not found Jul 23 15:13:02.997393 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.997141 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 15:13:02.997393 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.997157 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758d7fd447-qkrz6: secret "image-registry-tls" not found Jul 23 15:13:02.997393 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:02.997192 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls podName:f9b322c9-ea12-44a0-994a-b481ef39e4e3 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:03.997178263 +0000 UTC m=+37.613225509 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls") pod "image-registry-758d7fd447-qkrz6" (UID: "f9b322c9-ea12-44a0-994a-b481ef39e4e3") : secret "image-registry-tls" not found Jul 23 15:13:03.098285 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:03.098247 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:03.098488 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:03.098420 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 15:13:03.098488 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:03.098482 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls podName:5e6d6e02-f435-4ccd-9b43-5c4f79d86b50 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:04.09846684 +0000 UTC m=+37.714514082 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls") pod "dns-default-whcfg" (UID: "5e6d6e02-f435-4ccd-9b43-5c4f79d86b50") : secret "dns-default-metrics-tls" not found Jul 23 15:13:03.916671 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:03.916637 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8sl9b"] Jul 23 15:13:03.926636 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:03.926606 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x"] Jul 23 15:13:03.944385 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:03.943846 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-pnzkz"] Jul 23 15:13:03.959787 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:03.959757 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7"] Jul 23 15:13:03.972698 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:03.972672 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl"] Jul 23 15:13:04.004863 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.002564 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg"] Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.010334 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.010417 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.010511 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:04.010528 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle podName:f97ae7a3-b422-44a9-8942-752524470865 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:06.010507539 +0000 UTC m=+39.626554784 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle") pod "router-default-74d5f4575d-4d249" (UID: "f97ae7a3-b422-44a9-8942-752524470865") : configmap references non-existent config key: service-ca.crt Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.010576 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-v9g2q\" (UID: \"1b97d33c-4f23-4761-8dbe-7579d7b61a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:04.010600 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.010607 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:04.010657 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs podName:f97ae7a3-b422-44a9-8942-752524470865 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:06.010641659 +0000 UTC m=+39.626688909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs") pod "router-default-74d5f4575d-4d249" (UID: "f97ae7a3-b422-44a9-8942-752524470865") : secret "router-metrics-certs-default" not found Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.010340 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-tnxg9"] Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:04.010722 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:04.010734 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758d7fd447-qkrz6: secret "image-registry-tls" not found Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:04.010739 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:04.010765 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls podName:f9b322c9-ea12-44a0-994a-b481ef39e4e3 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:06.010754693 +0000 UTC m=+39.626801948 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls") pod "image-registry-758d7fd447-qkrz6" (UID: "f9b322c9-ea12-44a0-994a-b481ef39e4e3") : secret "image-registry-tls" not found Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:04.010802 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls podName:45875b2d-d207-49c6-8b86-aa50e0198929 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:06.010771453 +0000 UTC m=+39.626818699 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9mmv8" (UID: "45875b2d-d207-49c6-8b86-aa50e0198929") : secret "cluster-monitoring-operator-tls" not found Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:04.010854 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 15:13:04.011328 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:04.010885 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls podName:1b97d33c-4f23-4761-8dbe-7579d7b61a5e nodeName:}" failed. No retries permitted until 2026-07-23 15:13:06.010875777 +0000 UTC m=+39.626923022 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-v9g2q" (UID: "1b97d33c-4f23-4761-8dbe-7579d7b61a5e") : secret "samples-operator-tls" not found Jul 23 15:13:04.012158 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.011891 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-jhn9s"] Jul 23 15:13:04.015431 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.015412 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt"] Jul 23 15:13:04.046602 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.046504 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-s6czh"] Jul 23 15:13:04.112004 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.111911 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:04.112141 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:04.112078 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 15:13:04.112141 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:04.112134 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls podName:5e6d6e02-f435-4ccd-9b43-5c4f79d86b50 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:06.112120521 +0000 UTC m=+39.728167763 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls") pod "dns-default-whcfg" (UID: "5e6d6e02-f435-4ccd-9b43-5c4f79d86b50") : secret "dns-default-metrics-tls" not found Jul 23 15:13:04.362909 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.362830 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6"] Jul 23 15:13:04.366543 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.366522 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" Jul 23 15:13:04.369987 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.369954 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 23 15:13:04.370221 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.370147 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bbvw5\"" Jul 23 15:13:04.370559 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.370456 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 23 15:13:04.375596 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.375576 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6"] Jul 23 15:13:04.516907 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.516870 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/aa8c3277-5337-4477-adf1-f2917985d495-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-9ljm6\" (UID: \"aa8c3277-5337-4477-adf1-f2917985d495\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" Jul 23 15:13:04.517366 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.516917 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9ljm6\" (UID: \"aa8c3277-5337-4477-adf1-f2917985d495\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" Jul 23 15:13:04.610434 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.610117 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" event={"ID":"36dc10ed-f042-4994-8241-113f098485d6","Type":"ContainerStarted","Data":"da581d49220705ee593fe1f28ad58f301c34d7cfefcad3fa5996197c30fd0d5c"} Jul 23 15:13:04.613164 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.613044 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-s6czh" event={"ID":"5522141f-9f92-48ad-a954-7c0bff719ebd","Type":"ContainerStarted","Data":"3e6652b687fbc7b4c4aff749d7a705d12c2a24b23c94806b4846f30e961fca91"} Jul 23 15:13:04.616123 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.616057 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7" event={"ID":"78c82b4f-1367-497e-8767-7372b0c2d9e4","Type":"ContainerStarted","Data":"462af2cedc31c3a0c0e4993a48f13585495cd33f45367a2529c542a336f901d6"} Jul 23 15:13:04.619177 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.617608 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/aa8c3277-5337-4477-adf1-f2917985d495-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-9ljm6\" (UID: \"aa8c3277-5337-4477-adf1-f2917985d495\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" Jul 23 15:13:04.619177 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.617644 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9ljm6\" (UID: \"aa8c3277-5337-4477-adf1-f2917985d495\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" Jul 23 15:13:04.619177 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:04.617891 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 15:13:04.619177 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:04.617953 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert podName:aa8c3277-5337-4477-adf1-f2917985d495 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:05.117933773 +0000 UTC m=+38.733981032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9ljm6" (UID: "aa8c3277-5337-4477-adf1-f2917985d495") : secret "networking-console-plugin-cert" not found Jul 23 15:13:04.619177 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.619121 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/aa8c3277-5337-4477-adf1-f2917985d495-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-9ljm6\" (UID: \"aa8c3277-5337-4477-adf1-f2917985d495\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" Jul 23 15:13:04.621745 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.621672 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8sl9b" event={"ID":"e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9","Type":"ContainerStarted","Data":"38c4e2fe060dbac99538b476c34e0675b962b19030221fc96b224e724146c559"} Jul 23 15:13:04.623702 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.623628 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tnxg9" event={"ID":"38a36f07-0e32-46d6-9a99-2bd74ed87244","Type":"ContainerStarted","Data":"7f40b4f54eb4377f4d726128c8b59b7307c88803ad87275e1c855abd2925bbc2"} Jul 23 15:13:04.629933 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.629881 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" event={"ID":"71d3e351-b9cf-47fb-89b7-d69324c7a146","Type":"ContainerStarted","Data":"4b037bf7ed560b7f1e16654295b6e1e7098120351baf28966153f6c2f1e84fc4"} Jul 23 15:13:04.633555 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.633495 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-pnzkz" event={"ID":"c59394de-e921-4c8e-ab5c-d310242a8149","Type":"ContainerStarted","Data":"e399fdb1be4c284b5d1fe3f2921fcb29aa9abfa88d765d4ea897444c434cc556"} Jul 23 15:13:04.641213 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.640224 2570 generic.go:356] "Generic (PLEG): container finished" podID="8028e717-3208-45d4-a49f-117180037b85" containerID="9bb6077f7465743ae953201d733b1fe9aaf084983ece6734336ddbdfde6aa97c" exitCode=0 Jul 23 15:13:04.641213 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.640318 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bhpc" event={"ID":"8028e717-3208-45d4-a49f-117180037b85","Type":"ContainerDied","Data":"9bb6077f7465743ae953201d733b1fe9aaf084983ece6734336ddbdfde6aa97c"} Jul 23 15:13:04.647966 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.647898 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" event={"ID":"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32","Type":"ContainerStarted","Data":"67ae2f6e772e0d986bb2825685012620762e4b7cec7b37e6b9c3f30161af05a1"} Jul 23 15:13:04.652018 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.651968 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" event={"ID":"fee87863-6596-4de1-bf68-002c95fb789a","Type":"ContainerStarted","Data":"d8f17f6fa987970e67a1a956e6d4e636e1497f755b1a0f313b1eae6f41a8aa70"} Jul 23 15:13:04.657243 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:04.657220 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-jhn9s" event={"ID":"48561ae8-8b4f-4367-a2ef-20584d90b8e4","Type":"ContainerStarted","Data":"2f61e733048ebe120c5719aa58b2db037ee122beefc998a7e4e1849b8876e02f"} Jul 23 15:13:05.127482 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:05.127394 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9ljm6\" (UID: \"aa8c3277-5337-4477-adf1-f2917985d495\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" Jul 23 15:13:05.128040 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:05.127589 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 15:13:05.128040 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:05.127664 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert podName:aa8c3277-5337-4477-adf1-f2917985d495 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:06.127644354 +0000 UTC m=+39.743691599 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9ljm6" (UID: "aa8c3277-5337-4477-adf1-f2917985d495") : secret "networking-console-plugin-cert" not found Jul 23 15:13:05.710505 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:05.710458 2570 generic.go:356] "Generic (PLEG): container finished" podID="8028e717-3208-45d4-a49f-117180037b85" containerID="991fe79058391ef921c891f1052aae7fc0088a20e5c5334e6fcf9be94b228ee6" exitCode=0 Jul 23 15:13:05.710691 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:05.710567 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bhpc" event={"ID":"8028e717-3208-45d4-a49f-117180037b85","Type":"ContainerDied","Data":"991fe79058391ef921c891f1052aae7fc0088a20e5c5334e6fcf9be94b228ee6"} Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:06.042628 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:06.042762 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:06.042781 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:06.042797 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-v9g2q\" (UID: \"1b97d33c-4f23-4761-8dbe-7579d7b61a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:06.042841 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:06.042861 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls podName:45875b2d-d207-49c6-8b86-aa50e0198929 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:10.042839714 +0000 UTC m=+43.658886960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9mmv8" (UID: "45875b2d-d207-49c6-8b86-aa50e0198929") : secret "cluster-monitoring-operator-tls" not found Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:06.042929 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:06.042937 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:06.042991 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:06.042990 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs podName:f97ae7a3-b422-44a9-8942-752524470865 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:10.042973532 +0000 UTC m=+43.659020787 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs") pod "router-default-74d5f4575d-4d249" (UID: "f97ae7a3-b422-44a9-8942-752524470865") : secret "router-metrics-certs-default" not found Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:06.042935 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:06.043038 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758d7fd447-qkrz6: secret "image-registry-tls" not found Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:06.043052 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls podName:1b97d33c-4f23-4761-8dbe-7579d7b61a5e nodeName:}" failed. No retries permitted until 2026-07-23 15:13:10.043041179 +0000 UTC m=+43.659088425 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-v9g2q" (UID: "1b97d33c-4f23-4761-8dbe-7579d7b61a5e") : secret "samples-operator-tls" not found Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:06.043071 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle podName:f97ae7a3-b422-44a9-8942-752524470865 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:10.043063079 +0000 UTC m=+43.659110327 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle") pod "router-default-74d5f4575d-4d249" (UID: "f97ae7a3-b422-44a9-8942-752524470865") : configmap references non-existent config key: service-ca.crt Jul 23 15:13:06.043169 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:06.043087 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls podName:f9b322c9-ea12-44a0-994a-b481ef39e4e3 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:10.043079591 +0000 UTC m=+43.659126888 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls") pod "image-registry-758d7fd447-qkrz6" (UID: "f9b322c9-ea12-44a0-994a-b481ef39e4e3") : secret "image-registry-tls" not found Jul 23 15:13:06.144604 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:06.143730 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:06.144604 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:06.143810 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9ljm6\" (UID: \"aa8c3277-5337-4477-adf1-f2917985d495\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" Jul 23 15:13:06.144604 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:06.144017 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 15:13:06.144604 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:06.144081 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert podName:aa8c3277-5337-4477-adf1-f2917985d495 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:08.144061801 +0000 UTC m=+41.760109050 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9ljm6" (UID: "aa8c3277-5337-4477-adf1-f2917985d495") : secret "networking-console-plugin-cert" not found Jul 23 15:13:06.144604 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:06.144511 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 15:13:06.144604 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:06.144561 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls podName:5e6d6e02-f435-4ccd-9b43-5c4f79d86b50 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:10.14454566 +0000 UTC m=+43.760592907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls") pod "dns-default-whcfg" (UID: "5e6d6e02-f435-4ccd-9b43-5c4f79d86b50") : secret "dns-default-metrics-tls" not found Jul 23 15:13:06.771084 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:06.770151 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bhpc" event={"ID":"8028e717-3208-45d4-a49f-117180037b85","Type":"ContainerStarted","Data":"7e899e16551016b46536e7bb4d4bf8f3c51d0b9d66591d78271f5665a3bc84f2"} Jul 23 15:13:06.806235 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:06.804786 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4bhpc" podStartSLOduration=9.82624539 podStartE2EDuration="38.80476277s" totalImagesPullingTime="28.97851738s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:12:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:06.803244633 +0000 UTC m=+40.419291898" watchObservedRunningTime="2026-07-23 15:13:06.80476277 +0000 UTC m=+40.420810038" Jul 23 15:13:08.168680 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:08.168637 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9ljm6\" (UID: \"aa8c3277-5337-4477-adf1-f2917985d495\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" Jul 23 15:13:08.169105 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:08.168857 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 15:13:08.169105 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:08.168920 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert podName:aa8c3277-5337-4477-adf1-f2917985d495 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:12.168901703 +0000 UTC m=+45.784948951 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9ljm6" (UID: "aa8c3277-5337-4477-adf1-f2917985d495") : secret "networking-console-plugin-cert" not found Jul 23 15:13:10.086620 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:10.086561 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:10.087104 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:10.086695 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:10.087104 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:10.086729 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-v9g2q\" (UID: \"1b97d33c-4f23-4761-8dbe-7579d7b61a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" Jul 23 15:13:10.087104 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:10.086733 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 15:13:10.087104 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:10.086761 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:10.087104 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:10.086820 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls podName:45875b2d-d207-49c6-8b86-aa50e0198929 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:18.086798747 +0000 UTC m=+51.702846192 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9mmv8" (UID: "45875b2d-d207-49c6-8b86-aa50e0198929") : secret "cluster-monitoring-operator-tls" not found Jul 23 15:13:10.087104 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:10.086849 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 15:13:10.087104 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:10.086863 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 15:13:10.087104 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:10.086861 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:10.087104 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:10.086875 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758d7fd447-qkrz6: secret "image-registry-tls" not found Jul 23 15:13:10.087104 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:10.086897 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 15:13:10.087104 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:10.086902 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs podName:f97ae7a3-b422-44a9-8942-752524470865 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:18.086887031 +0000 UTC m=+51.702934288 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs") pod "router-default-74d5f4575d-4d249" (UID: "f97ae7a3-b422-44a9-8942-752524470865") : secret "router-metrics-certs-default" not found Jul 23 15:13:10.087104 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:10.086964 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls podName:f9b322c9-ea12-44a0-994a-b481ef39e4e3 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:18.086952092 +0000 UTC m=+51.702999333 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls") pod "image-registry-758d7fd447-qkrz6" (UID: "f9b322c9-ea12-44a0-994a-b481ef39e4e3") : secret "image-registry-tls" not found Jul 23 15:13:10.087104 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:10.086984 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle podName:f97ae7a3-b422-44a9-8942-752524470865 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:18.086976093 +0000 UTC m=+51.703023336 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle") pod "router-default-74d5f4575d-4d249" (UID: "f97ae7a3-b422-44a9-8942-752524470865") : configmap references non-existent config key: service-ca.crt Jul 23 15:13:10.087104 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:10.086999 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls podName:1b97d33c-4f23-4761-8dbe-7579d7b61a5e nodeName:}" failed. No retries permitted until 2026-07-23 15:13:18.086992476 +0000 UTC m=+51.703039722 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-v9g2q" (UID: "1b97d33c-4f23-4761-8dbe-7579d7b61a5e") : secret "samples-operator-tls" not found Jul 23 15:13:10.187691 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:10.187656 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:10.187847 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:10.187821 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 15:13:10.187896 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:10.187884 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls podName:5e6d6e02-f435-4ccd-9b43-5c4f79d86b50 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:18.187868736 +0000 UTC m=+51.803915978 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls") pod "dns-default-whcfg" (UID: "5e6d6e02-f435-4ccd-9b43-5c4f79d86b50") : secret "dns-default-metrics-tls" not found Jul 23 15:13:12.204628 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:12.204590 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9ljm6\" (UID: \"aa8c3277-5337-4477-adf1-f2917985d495\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" Jul 23 15:13:12.205103 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:12.204756 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 15:13:12.205103 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:12.204823 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert podName:aa8c3277-5337-4477-adf1-f2917985d495 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:20.204806098 +0000 UTC m=+53.820853339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9ljm6" (UID: "aa8c3277-5337-4477-adf1-f2917985d495") : secret "networking-console-plugin-cert" not found Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.160717 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.160802 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.160900 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.160928 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-v9g2q\" (UID: \"1b97d33c-4f23-4761-8dbe-7579d7b61a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.160969 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:18.161125 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:18.161144 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758d7fd447-qkrz6: secret "image-registry-tls" not found Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:18.161193 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls podName:f9b322c9-ea12-44a0-994a-b481ef39e4e3 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:34.161176152 +0000 UTC m=+67.777223413 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls") pod "image-registry-758d7fd447-qkrz6" (UID: "f9b322c9-ea12-44a0-994a-b481ef39e4e3") : secret "image-registry-tls" not found Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:18.161269 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle podName:f97ae7a3-b422-44a9-8942-752524470865 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:34.161258993 +0000 UTC m=+67.777306236 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle") pod "router-default-74d5f4575d-4d249" (UID: "f97ae7a3-b422-44a9-8942-752524470865") : configmap references non-existent config key: service-ca.crt Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:18.161354 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:18.161392 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls podName:45875b2d-d207-49c6-8b86-aa50e0198929 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:34.161379419 +0000 UTC m=+67.777426667 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-9mmv8" (UID: "45875b2d-d207-49c6-8b86-aa50e0198929") : secret "cluster-monitoring-operator-tls" not found Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:18.161417 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:18.161464 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:18.161476 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs podName:f97ae7a3-b422-44a9-8942-752524470865 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:34.16145872 +0000 UTC m=+67.777505961 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs") pod "router-default-74d5f4575d-4d249" (UID: "f97ae7a3-b422-44a9-8942-752524470865") : secret "router-metrics-certs-default" not found Jul 23 15:13:18.161543 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:18.161503 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls podName:1b97d33c-4f23-4761-8dbe-7579d7b61a5e nodeName:}" failed. No retries permitted until 2026-07-23 15:13:34.161490279 +0000 UTC m=+67.777537534 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-v9g2q" (UID: "1b97d33c-4f23-4761-8dbe-7579d7b61a5e") : secret "samples-operator-tls" not found Jul 23 15:13:18.262544 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.262465 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:18.262709 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:18.262689 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 15:13:18.262788 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:18.262764 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls podName:5e6d6e02-f435-4ccd-9b43-5c4f79d86b50 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:34.262745069 +0000 UTC m=+67.878792314 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls") pod "dns-default-whcfg" (UID: "5e6d6e02-f435-4ccd-9b43-5c4f79d86b50") : secret "dns-default-metrics-tls" not found Jul 23 15:13:18.811803 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.811762 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-jhn9s" event={"ID":"48561ae8-8b4f-4367-a2ef-20584d90b8e4","Type":"ContainerStarted","Data":"d83d2bfde38da03309d09c3db6a69db93fcfc2f3d6128052046e0996deb1fcd4"} Jul 23 15:13:18.813504 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.813474 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" event={"ID":"36dc10ed-f042-4994-8241-113f098485d6","Type":"ContainerStarted","Data":"5eed8e40ed0c72aed3853011093b7b0ae15538b5b8d94d04333f203ff93cd534"} Jul 23 15:13:18.815266 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.815243 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s6czh_5522141f-9f92-48ad-a954-7c0bff719ebd/console-operator/0.log" Jul 23 15:13:18.815463 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.815319 2570 generic.go:356] "Generic (PLEG): container finished" podID="5522141f-9f92-48ad-a954-7c0bff719ebd" containerID="44a2d9b0b6848af18248d87237ebb814542b740ed2726ad268368c9f5f4c65d6" exitCode=255 Jul 23 15:13:18.815463 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.815396 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-s6czh" event={"ID":"5522141f-9f92-48ad-a954-7c0bff719ebd","Type":"ContainerDied","Data":"44a2d9b0b6848af18248d87237ebb814542b740ed2726ad268368c9f5f4c65d6"} Jul 23 15:13:18.815594 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.815575 2570 scope.go:117] "RemoveContainer" containerID="44a2d9b0b6848af18248d87237ebb814542b740ed2726ad268368c9f5f4c65d6" Jul 23 15:13:18.817760 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.817738 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7" event={"ID":"78c82b4f-1367-497e-8767-7372b0c2d9e4","Type":"ContainerStarted","Data":"f1ef9eb7f207a6a7fe2784c112128bdbfbefff06a9019d9d4697e0a8fb9ad4ab"} Jul 23 15:13:18.819791 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.819770 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8sl9b" event={"ID":"e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9","Type":"ContainerStarted","Data":"bf3cdd3f29692afd43f10f1ee7c2ca7e9bdf15589a81e1d0478139b4bf6705a7"} Jul 23 15:13:18.821346 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.821316 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tnxg9" event={"ID":"38a36f07-0e32-46d6-9a99-2bd74ed87244","Type":"ContainerStarted","Data":"d377bdd9c81eadfb54891d8b3e244afa048e68435c0086304694a03772ee9fbc"} Jul 23 15:13:18.822765 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.822746 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" event={"ID":"71d3e351-b9cf-47fb-89b7-d69324c7a146","Type":"ContainerStarted","Data":"c4f31ec6c69b376e2f14d67a63e9d7c2cc9ad369f728867b3c4404547049102d"} Jul 23 15:13:18.824381 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.824358 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-pnzkz" event={"ID":"c59394de-e921-4c8e-ab5c-d310242a8149","Type":"ContainerStarted","Data":"683ad0353defe6ac92512dd9941f888fdb8a49af005fda0b2e09240d5d51ee48"} Jul 23 15:13:18.826495 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.826474 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" event={"ID":"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32","Type":"ContainerStarted","Data":"2ea5b3e1cb89456770613115e008d95bf6cc54730e3f60c3f67279b987d3559d"} Jul 23 15:13:18.826591 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.826507 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" event={"ID":"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32","Type":"ContainerStarted","Data":"a9afd12a8e36145f41c0c14613ef480996a899ff2db1c79ad6f317b2c4ff5f85"} Jul 23 15:13:18.826591 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.826524 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" event={"ID":"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32","Type":"ContainerStarted","Data":"3c8e6881ef54f4edeb7d73b86cf4894554c05443e9354111a7d398864f29fe14"} Jul 23 15:13:18.828172 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.828151 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" event={"ID":"fee87863-6596-4de1-bf68-002c95fb789a","Type":"ContainerStarted","Data":"c2bd6ab6ea8a966df4c771a7e7c2bbed5258d4b9108546685785e4630a9cf6d4"} Jul 23 15:13:18.828795 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.828780 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" Jul 23 15:13:18.830654 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.830636 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" Jul 23 15:13:18.846992 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.846937 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-jhn9s" podStartSLOduration=20.997600579 podStartE2EDuration="34.846915877s" totalImagesPullingTime="13.849315298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:18.841602137 +0000 UTC m=+52.457649393" watchObservedRunningTime="2026-07-23 15:13:18.846915877 +0000 UTC m=+52.462963134" Jul 23 15:13:18.882840 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.882798 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-pnzkz" podStartSLOduration=27.937380910999998 podStartE2EDuration="34.882780783s" totalImagesPullingTime="6.945399872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:18.881852496 +0000 UTC m=+52.497899759" watchObservedRunningTime="2026-07-23 15:13:18.882780783 +0000 UTC m=+52.498828048" Jul 23 15:13:18.907166 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.907110 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f95d4c776-cj58x" podStartSLOduration=11.914586268 podStartE2EDuration="25.907088056s" totalImagesPullingTime="13.992501788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:18.906894283 +0000 UTC m=+52.522941548" watchObservedRunningTime="2026-07-23 15:13:18.907088056 +0000 UTC m=+52.523135302" Jul 23 15:13:18.930113 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.930038 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8sl9b" podStartSLOduration=2.939407196 podStartE2EDuration="16.930014616s" totalImagesPullingTime="13.99060742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:18.929941766 +0000 UTC m=+52.545989030" watchObservedRunningTime="2026-07-23 15:13:18.930014616 +0000 UTC m=+52.546061882" Jul 23 15:13:18.953849 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.953788 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" podStartSLOduration=12.104881046 podStartE2EDuration="25.953763542s" totalImagesPullingTime="13.848882496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:18.951981348 +0000 UTC m=+52.568028614" watchObservedRunningTime="2026-07-23 15:13:18.953763542 +0000 UTC m=+52.569810807" Jul 23 15:13:18.973963 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.973896 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tnxg9" podStartSLOduration=21.127022644 podStartE2EDuration="34.973872634s" totalImagesPullingTime="13.84684999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:18.972487685 +0000 UTC m=+52.588534950" watchObservedRunningTime="2026-07-23 15:13:18.973872634 +0000 UTC m=+52.589919893" Jul 23 15:13:18.992452 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:18.992403 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7cd494ddd6-c8sf7" podStartSLOduration=12.143837067 podStartE2EDuration="25.99238453s" totalImagesPullingTime="13.848547463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:18.991511768 +0000 UTC m=+52.607559032" watchObservedRunningTime="2026-07-23 15:13:18.99238453 +0000 UTC m=+52.608431794" Jul 23 15:13:19.035636 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.035577 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" podStartSLOduration=21.189104561 podStartE2EDuration="35.035555429s" totalImagesPullingTime="13.846450868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:19.03392336 +0000 UTC m=+52.649970626" watchObservedRunningTime="2026-07-23 15:13:19.035555429 +0000 UTC m=+52.651602698" Jul 23 15:13:19.056279 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.056214 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" podStartSLOduration=21.206131045 podStartE2EDuration="35.056180008s" totalImagesPullingTime="13.850048963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:19.054513081 +0000 UTC m=+52.670560346" watchObservedRunningTime="2026-07-23 15:13:19.056180008 +0000 UTC m=+52.672227271" Jul 23 15:13:19.375541 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.375506 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fwzgb"] Jul 23 15:13:19.406256 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.406217 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fwzgb"] Jul 23 15:13:19.406415 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.406377 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:19.409418 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.409388 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 15:13:19.410028 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.410005 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 15:13:19.410190 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.410167 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m6wjn\"" Jul 23 15:13:19.474570 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.474534 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2v9z\" (UniqueName: \"kubernetes.io/projected/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-api-access-x2v9z\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:19.474721 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.474686 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:19.474765 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.474754 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-data-volume\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:19.474814 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.474800 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-crio-socket\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:19.474890 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.474871 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:19.576395 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.576354 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:19.576588 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.576482 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2v9z\" (UniqueName: \"kubernetes.io/projected/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-api-access-x2v9z\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:19.576588 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:19.576528 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 15:13:19.576588 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.576567 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:19.576753 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:19.576599 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls podName:9c43bfcd-1a27-4f40-a0c8-50e90817eb85 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:20.07658026 +0000 UTC m=+53.692627524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fwzgb" (UID: "9c43bfcd-1a27-4f40-a0c8-50e90817eb85") : secret "insights-runtime-extractor-tls" not found Jul 23 15:13:19.576753 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:19.576617 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 15:13:19.576753 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.576644 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-data-volume\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:19.576753 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:19.576652 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm podName:9c43bfcd-1a27-4f40-a0c8-50e90817eb85 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:20.076641145 +0000 UTC m=+53.692688388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fwzgb" (UID: "9c43bfcd-1a27-4f40-a0c8-50e90817eb85") : configmap "kube-rbac-proxy" not found Jul 23 15:13:19.576753 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.576695 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-crio-socket\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:19.577019 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.576794 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-crio-socket\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:19.577019 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.576990 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-data-volume\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:19.587362 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.587318 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2v9z\" (UniqueName: \"kubernetes.io/projected/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-api-access-x2v9z\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:19.832895 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.832822 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s6czh_5522141f-9f92-48ad-a954-7c0bff719ebd/console-operator/1.log" Jul 23 15:13:19.833282 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.833265 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s6czh_5522141f-9f92-48ad-a954-7c0bff719ebd/console-operator/0.log" Jul 23 15:13:19.833360 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.833328 2570 generic.go:356] "Generic (PLEG): container finished" podID="5522141f-9f92-48ad-a954-7c0bff719ebd" containerID="bb15571ec30d7e1f2121177d5dc7f6e0e4271ba3b555e400e4d7b31599b68746" exitCode=255 Jul 23 15:13:19.833468 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.833432 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-s6czh" event={"ID":"5522141f-9f92-48ad-a954-7c0bff719ebd","Type":"ContainerDied","Data":"bb15571ec30d7e1f2121177d5dc7f6e0e4271ba3b555e400e4d7b31599b68746"} Jul 23 15:13:19.833617 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.833507 2570 scope.go:117] "RemoveContainer" containerID="44a2d9b0b6848af18248d87237ebb814542b740ed2726ad268368c9f5f4c65d6" Jul 23 15:13:19.833680 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:19.833654 2570 scope.go:117] "RemoveContainer" containerID="bb15571ec30d7e1f2121177d5dc7f6e0e4271ba3b555e400e4d7b31599b68746" Jul 23 15:13:19.833876 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:19.833845 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-s6czh_openshift-console-operator(5522141f-9f92-48ad-a954-7c0bff719ebd)\"" pod="openshift-console-operator/console-operator-565b7b569d-s6czh" podUID="5522141f-9f92-48ad-a954-7c0bff719ebd" Jul 23 15:13:20.083087 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:20.082995 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:20.083255 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:20.083131 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 15:13:20.083255 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:20.083181 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:20.083255 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:20.083195 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls podName:9c43bfcd-1a27-4f40-a0c8-50e90817eb85 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:21.083179821 +0000 UTC m=+54.699227063 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fwzgb" (UID: "9c43bfcd-1a27-4f40-a0c8-50e90817eb85") : secret "insights-runtime-extractor-tls" not found Jul 23 15:13:20.083255 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:20.083237 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 15:13:20.083489 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:20.083282 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm podName:9c43bfcd-1a27-4f40-a0c8-50e90817eb85 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:21.083270305 +0000 UTC m=+54.699317547 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fwzgb" (UID: "9c43bfcd-1a27-4f40-a0c8-50e90817eb85") : configmap "kube-rbac-proxy" not found Jul 23 15:13:20.284631 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:20.284594 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9ljm6\" (UID: \"aa8c3277-5337-4477-adf1-f2917985d495\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" Jul 23 15:13:20.284778 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:20.284716 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 15:13:20.284778 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:20.284770 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert podName:aa8c3277-5337-4477-adf1-f2917985d495 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:36.284756968 +0000 UTC m=+69.900804211 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-9ljm6" (UID: "aa8c3277-5337-4477-adf1-f2917985d495") : secret "networking-console-plugin-cert" not found Jul 23 15:13:20.838044 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:20.838020 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s6czh_5522141f-9f92-48ad-a954-7c0bff719ebd/console-operator/1.log" Jul 23 15:13:20.838453 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:20.838374 2570 scope.go:117] "RemoveContainer" containerID="bb15571ec30d7e1f2121177d5dc7f6e0e4271ba3b555e400e4d7b31599b68746" Jul 23 15:13:20.838537 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:20.838518 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-s6czh_openshift-console-operator(5522141f-9f92-48ad-a954-7c0bff719ebd)\"" pod="openshift-console-operator/console-operator-565b7b569d-s6czh" podUID="5522141f-9f92-48ad-a954-7c0bff719ebd" Jul 23 15:13:21.091841 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:21.091755 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:21.091996 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:21.091848 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:21.091996 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:21.091889 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 15:13:21.091996 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:21.091961 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 15:13:21.092377 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:21.092355 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm podName:9c43bfcd-1a27-4f40-a0c8-50e90817eb85 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:23.092319824 +0000 UTC m=+56.708367080 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fwzgb" (UID: "9c43bfcd-1a27-4f40-a0c8-50e90817eb85") : configmap "kube-rbac-proxy" not found Jul 23 15:13:21.093140 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:21.093111 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls podName:9c43bfcd-1a27-4f40-a0c8-50e90817eb85 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:23.093086558 +0000 UTC m=+56.709133815 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fwzgb" (UID: "9c43bfcd-1a27-4f40-a0c8-50e90817eb85") : secret "insights-runtime-extractor-tls" not found Jul 23 15:13:21.370277 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:21.370249 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mwbjc_15c40f6c-249b-4f48-a8ae-f8b6da077e0e/dns-node-resolver/0.log" Jul 23 15:13:22.571206 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:22.571178 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k57jv_8322eaa3-685b-44b5-834f-ae04b6ec6566/node-ca/0.log" Jul 23 15:13:22.639519 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:22.639482 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:22.639650 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:22.639546 2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:22.639882 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:22.639869 2570 scope.go:117] "RemoveContainer" containerID="bb15571ec30d7e1f2121177d5dc7f6e0e4271ba3b555e400e4d7b31599b68746" Jul 23 15:13:22.640056 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:22.640038 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-s6czh_openshift-console-operator(5522141f-9f92-48ad-a954-7c0bff719ebd)\"" pod="openshift-console-operator/console-operator-565b7b569d-s6czh" podUID="5522141f-9f92-48ad-a954-7c0bff719ebd" Jul 23 15:13:23.110448 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:23.110402 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:23.110645 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:23.110487 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:23.110645 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:23.110530 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 15:13:23.110645 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:23.110610 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm podName:9c43bfcd-1a27-4f40-a0c8-50e90817eb85 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:27.110594896 +0000 UTC m=+60.726642139 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fwzgb" (UID: "9c43bfcd-1a27-4f40-a0c8-50e90817eb85") : configmap "kube-rbac-proxy" not found Jul 23 15:13:23.110645 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:23.110632 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 15:13:23.110811 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:23.110689 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls podName:9c43bfcd-1a27-4f40-a0c8-50e90817eb85 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:27.110676426 +0000 UTC m=+60.726723668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fwzgb" (UID: "9c43bfcd-1a27-4f40-a0c8-50e90817eb85") : secret "insights-runtime-extractor-tls" not found Jul 23 15:13:23.171056 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:23.171023 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8sl9b_e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9/serve-healthcheck-canary/0.log" Jul 23 15:13:27.147830 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:27.147772 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:27.148325 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:27.147886 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:27.148325 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:27.147901 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 15:13:27.148325 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:27.147963 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm podName:9c43bfcd-1a27-4f40-a0c8-50e90817eb85 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:35.147948707 +0000 UTC m=+68.763995949 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fwzgb" (UID: "9c43bfcd-1a27-4f40-a0c8-50e90817eb85") : configmap "kube-rbac-proxy" not found Jul 23 15:13:27.148325 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:27.148019 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 15:13:27.148325 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:27.148076 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls podName:9c43bfcd-1a27-4f40-a0c8-50e90817eb85 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:35.148060037 +0000 UTC m=+68.764107279 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fwzgb" (UID: "9c43bfcd-1a27-4f40-a0c8-50e90817eb85") : secret "insights-runtime-extractor-tls" not found Jul 23 15:13:28.600949 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:28.600919 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cdjsd" Jul 23 15:13:33.815477 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:33.815440 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs\") pod \"network-metrics-daemon-n5624\" (UID: \"7b189e73-6f40-467e-9bf1-5505c5a7194c\") " pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:13:33.815887 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:33.815538 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:13:33.818471 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:33.818455 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 15:13:33.818779 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:33.818764 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 15:13:33.826529 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:33.826512 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 23 15:13:33.826586 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:33.826566 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs podName:7b189e73-6f40-467e-9bf1-5505c5a7194c nodeName:}" failed. No retries permitted until 2026-07-23 15:14:37.826548694 +0000 UTC m=+131.442595939 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs") pod "network-metrics-daemon-n5624" (UID: "7b189e73-6f40-467e-9bf1-5505c5a7194c") : secret "metrics-daemon-secret" not found Jul 23 15:13:33.828793 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:33.828770 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7cff11e5-8f10-4219-9ed9-76a3c70061c8-original-pull-secret\") pod \"global-pull-secret-syncer-wl9qs\" (UID: \"7cff11e5-8f10-4219-9ed9-76a3c70061c8\") " pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:13:33.916820 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:33.916788 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrxqw\" (UniqueName: \"kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw\") pod \"network-check-target-ltmrf\" (UID: \"5354f3d7-6d0a-48bf-b8c1-85949c45d9d2\") " pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:13:33.919251 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:33.919219 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrxqw\" (UniqueName: \"kubernetes.io/projected/5354f3d7-6d0a-48bf-b8c1-85949c45d9d2-kube-api-access-zrxqw\") pod \"network-check-target-ltmrf\" (UID: \"5354f3d7-6d0a-48bf-b8c1-85949c45d9d2\") " pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:13:33.943905 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:33.943882 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wl9qs" Jul 23 15:13:33.959444 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:33.959416 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-srpb5\"" Jul 23 15:13:33.967055 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:33.967034 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:13:34.081340 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.081236 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wl9qs"] Jul 23 15:13:34.084660 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:13:34.084629 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cff11e5_8f10_4219_9ed9_76a3c70061c8.slice/crio-8a240597eccdabd9fcb4e1d23f3f42c5ae89b0b4dee45af264fa9cc45dde5514 WatchSource:0}: Error finding container 8a240597eccdabd9fcb4e1d23f3f42c5ae89b0b4dee45af264fa9cc45dde5514: Status 404 returned error can't find the container with id 8a240597eccdabd9fcb4e1d23f3f42c5ae89b0b4dee45af264fa9cc45dde5514 Jul 23 15:13:34.108938 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.108914 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ltmrf"] Jul 23 15:13:34.111648 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:13:34.111623 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5354f3d7_6d0a_48bf_b8c1_85949c45d9d2.slice/crio-5ddf6e4ec80c1c134d8e5086a774a7e5b94abadac1319379b4ee2e7c279f5fc9 WatchSource:0}: Error finding container 5ddf6e4ec80c1c134d8e5086a774a7e5b94abadac1319379b4ee2e7c279f5fc9: Status 404 returned error can't find the container with id 5ddf6e4ec80c1c134d8e5086a774a7e5b94abadac1319379b4ee2e7c279f5fc9 Jul 23 15:13:34.219871 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.219836 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:34.220025 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.219916 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:34.220025 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.219937 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-v9g2q\" (UID: \"1b97d33c-4f23-4761-8dbe-7579d7b61a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" Jul 23 15:13:34.220025 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.219963 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:34.220025 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.219992 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:34.220637 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.220608 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f97ae7a3-b422-44a9-8942-752524470865-service-ca-bundle\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:34.222322 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.222272 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls\") pod \"image-registry-758d7fd447-qkrz6\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:34.222423 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.222383 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45875b2d-d207-49c6-8b86-aa50e0198929-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-9mmv8\" (UID: \"45875b2d-d207-49c6-8b86-aa50e0198929\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:34.222463 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.222426 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b97d33c-4f23-4761-8dbe-7579d7b61a5e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-v9g2q\" (UID: \"1b97d33c-4f23-4761-8dbe-7579d7b61a5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" Jul 23 15:13:34.222463 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.222426 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f97ae7a3-b422-44a9-8942-752524470865-metrics-certs\") pod \"router-default-74d5f4575d-4d249\" (UID: \"f97ae7a3-b422-44a9-8942-752524470865\") " pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:34.321085 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.321052 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:34.323253 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.323216 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e6d6e02-f435-4ccd-9b43-5c4f79d86b50-metrics-tls\") pod \"dns-default-whcfg\" (UID: \"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50\") " pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:34.392041 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.392011 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kqqgz\"" Jul 23 15:13:34.399406 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.399388 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" Jul 23 15:13:34.422957 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.422935 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zt8sw\"" Jul 23 15:13:34.430083 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.430059 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:34.448820 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.448796 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-khmmr\"" Jul 23 15:13:34.455920 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.455103 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wqjcr\"" Jul 23 15:13:34.456259 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.456237 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:34.462381 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.462306 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" Jul 23 15:13:34.528896 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.528817 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7fsth\"" Jul 23 15:13:34.537215 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.536967 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:34.572056 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.572019 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8"] Jul 23 15:13:34.580527 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:13:34.580495 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45875b2d_d207_49c6_8b86_aa50e0198929.slice/crio-2674f3845baa88aff931bee57a21e755408f0499439383410bb6553e403faeb7 WatchSource:0}: Error finding container 2674f3845baa88aff931bee57a21e755408f0499439383410bb6553e403faeb7: Status 404 returned error can't find the container with id 2674f3845baa88aff931bee57a21e755408f0499439383410bb6553e403faeb7 Jul 23 15:13:34.616414 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.616225 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-758d7fd447-qkrz6"] Jul 23 15:13:34.621717 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:13:34.621678 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b322c9_ea12_44a0_994a_b481ef39e4e3.slice/crio-1ea0815998a05c3c30cc55ef2aa3472f57c3f10fd268768df413c4921b9dc875 WatchSource:0}: Error finding container 1ea0815998a05c3c30cc55ef2aa3472f57c3f10fd268768df413c4921b9dc875: Status 404 returned error can't find the container with id 1ea0815998a05c3c30cc55ef2aa3472f57c3f10fd268768df413c4921b9dc875 Jul 23 15:13:34.639689 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.639657 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74d5f4575d-4d249"] Jul 23 15:13:34.643808 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:13:34.643745 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf97ae7a3_b422_44a9_8942_752524470865.slice/crio-c6cfa8f807072093ea67e10f5f2a767aa0e784ca001fda70071e09d224455ca8 WatchSource:0}: Error finding container c6cfa8f807072093ea67e10f5f2a767aa0e784ca001fda70071e09d224455ca8: Status 404 returned error can't find the container with id c6cfa8f807072093ea67e10f5f2a767aa0e784ca001fda70071e09d224455ca8 Jul 23 15:13:34.654579 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.654511 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q"] Jul 23 15:13:34.713832 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.713765 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-whcfg"] Jul 23 15:13:34.718070 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:13:34.718042 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e6d6e02_f435_4ccd_9b43_5c4f79d86b50.slice/crio-649e0efb26c52006c3db00f66bb3299d9e307a56bc24c47a05354be080cb04c2 WatchSource:0}: Error finding container 649e0efb26c52006c3db00f66bb3299d9e307a56bc24c47a05354be080cb04c2: Status 404 returned error can't find the container with id 649e0efb26c52006c3db00f66bb3299d9e307a56bc24c47a05354be080cb04c2 Jul 23 15:13:34.883320 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.883266 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-whcfg" event={"ID":"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50","Type":"ContainerStarted","Data":"649e0efb26c52006c3db00f66bb3299d9e307a56bc24c47a05354be080cb04c2"} Jul 23 15:13:34.884762 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.884738 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ltmrf" event={"ID":"5354f3d7-6d0a-48bf-b8c1-85949c45d9d2","Type":"ContainerStarted","Data":"e61fa56c6bca94db14320edcfda758e523b82f2ba159eeb04b7826660527725b"} Jul 23 15:13:34.884871 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.884777 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ltmrf" event={"ID":"5354f3d7-6d0a-48bf-b8c1-85949c45d9d2","Type":"ContainerStarted","Data":"5ddf6e4ec80c1c134d8e5086a774a7e5b94abadac1319379b4ee2e7c279f5fc9"} Jul 23 15:13:34.884871 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.884859 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:13:34.886263 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.886243 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74d5f4575d-4d249" event={"ID":"f97ae7a3-b422-44a9-8942-752524470865","Type":"ContainerStarted","Data":"d61700c9933bd944ae584d249b8b44d4a7746ef4c4a382fd99b16329a7e4b448"} Jul 23 15:13:34.886342 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.886279 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74d5f4575d-4d249" event={"ID":"f97ae7a3-b422-44a9-8942-752524470865","Type":"ContainerStarted","Data":"c6cfa8f807072093ea67e10f5f2a767aa0e784ca001fda70071e09d224455ca8"} Jul 23 15:13:34.887687 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.887666 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" event={"ID":"f9b322c9-ea12-44a0-994a-b481ef39e4e3","Type":"ContainerStarted","Data":"6c6ae522d9300332c062d56b123e57f873e5d5b46918554548bb8a4938dec3d3"} Jul 23 15:13:34.887789 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.887704 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" event={"ID":"f9b322c9-ea12-44a0-994a-b481ef39e4e3","Type":"ContainerStarted","Data":"1ea0815998a05c3c30cc55ef2aa3472f57c3f10fd268768df413c4921b9dc875"} Jul 23 15:13:34.887789 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.887728 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:13:34.888779 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.888755 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" event={"ID":"1b97d33c-4f23-4761-8dbe-7579d7b61a5e","Type":"ContainerStarted","Data":"e476189928687044bdc11ceb7375475bbdb293b762cf06e76d699d5d942d2d18"} Jul 23 15:13:34.889777 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.889759 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" event={"ID":"45875b2d-d207-49c6-8b86-aa50e0198929","Type":"ContainerStarted","Data":"2674f3845baa88aff931bee57a21e755408f0499439383410bb6553e403faeb7"} Jul 23 15:13:34.890759 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.890737 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wl9qs" event={"ID":"7cff11e5-8f10-4219-9ed9-76a3c70061c8","Type":"ContainerStarted","Data":"8a240597eccdabd9fcb4e1d23f3f42c5ae89b0b4dee45af264fa9cc45dde5514"} Jul 23 15:13:34.908214 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.908025 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ltmrf" podStartSLOduration=66.908006289 podStartE2EDuration="1m6.908006289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:34.90625859 +0000 UTC m=+68.522305882" watchObservedRunningTime="2026-07-23 15:13:34.908006289 +0000 UTC m=+68.524053554" Jul 23 15:13:34.936637 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.936571 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-74d5f4575d-4d249" podStartSLOduration=50.936549631 podStartE2EDuration="50.936549631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:34.934353908 +0000 UTC m=+68.550401177" watchObservedRunningTime="2026-07-23 15:13:34.936549631 +0000 UTC m=+68.552596897" Jul 23 15:13:34.966477 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:34.966414 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" podStartSLOduration=65.966392049 podStartE2EDuration="1m5.966392049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:34.964343691 +0000 UTC m=+68.580390957" watchObservedRunningTime="2026-07-23 15:13:34.966392049 +0000 UTC m=+68.582439313" Jul 23 15:13:35.227932 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:35.227838 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:35.227932 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:35.227928 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:35.228164 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:35.228087 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 15:13:35.228164 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:13:35.228150 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls podName:9c43bfcd-1a27-4f40-a0c8-50e90817eb85 nodeName:}" failed. No retries permitted until 2026-07-23 15:13:51.228132088 +0000 UTC m=+84.844179348 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fwzgb" (UID: "9c43bfcd-1a27-4f40-a0c8-50e90817eb85") : secret "insights-runtime-extractor-tls" not found Jul 23 15:13:35.228475 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:35.228447 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:35.456960 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:35.456917 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:35.459881 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:35.459860 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:35.896403 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:35.896345 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:35.898029 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:35.898010 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-74d5f4575d-4d249" Jul 23 15:13:36.339896 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:36.339806 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9ljm6\" (UID: \"aa8c3277-5337-4477-adf1-f2917985d495\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" Jul 23 15:13:36.342831 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:36.342805 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/aa8c3277-5337-4477-adf1-f2917985d495-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9ljm6\" (UID: \"aa8c3277-5337-4477-adf1-f2917985d495\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" Jul 23 15:13:36.478862 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:36.478830 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bbvw5\"" Jul 23 15:13:36.486445 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:36.486415 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" Jul 23 15:13:37.433473 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:37.433438 2570 scope.go:117] "RemoveContainer" containerID="bb15571ec30d7e1f2121177d5dc7f6e0e4271ba3b555e400e4d7b31599b68746" Jul 23 15:13:39.196339 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.196315 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6"] Jul 23 15:13:39.918738 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.918703 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" event={"ID":"aa8c3277-5337-4477-adf1-f2917985d495","Type":"ContainerStarted","Data":"07ff7fe731609124d91d656ae862ae0d2ae86c25d6c6c438dff4d860e7ff0cca"} Jul 23 15:13:39.921179 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.921158 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s6czh_5522141f-9f92-48ad-a954-7c0bff719ebd/console-operator/1.log" Jul 23 15:13:39.921350 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.921286 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-s6czh" event={"ID":"5522141f-9f92-48ad-a954-7c0bff719ebd","Type":"ContainerStarted","Data":"d471628a2c8d56251a694e8c446b3d1127df0057d3f00291958728a22f74d3cf"} Jul 23 15:13:39.921721 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.921648 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:39.924080 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.924057 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" event={"ID":"1b97d33c-4f23-4761-8dbe-7579d7b61a5e","Type":"ContainerStarted","Data":"a2f1552d01a2899bd4d551466d8bce94b99c3fde5ec89ddf5201f881509611bf"} Jul 23 15:13:39.924177 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.924105 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" event={"ID":"1b97d33c-4f23-4761-8dbe-7579d7b61a5e","Type":"ContainerStarted","Data":"076b293d204458940c02f2a7a627daf563f34fa8d8aec563fe3f998a857364ff"} Jul 23 15:13:39.927748 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.927720 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" event={"ID":"45875b2d-d207-49c6-8b86-aa50e0198929","Type":"ContainerStarted","Data":"b360af216bdcde2c874c4a49d8a5a04c829dff1d245bde58f118c75e7cf9b53f"} Jul 23 15:13:39.929032 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.928985 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-s6czh" Jul 23 15:13:39.930567 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.930543 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wl9qs" event={"ID":"7cff11e5-8f10-4219-9ed9-76a3c70061c8","Type":"ContainerStarted","Data":"e679b4691851bcd71378cd21c9235a8c5c2e4e08505513672db8901d2d1efeed"} Jul 23 15:13:39.932523 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.932501 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-whcfg" event={"ID":"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50","Type":"ContainerStarted","Data":"862461d622229de3f2322f7de7dd147d86968f9734835836cf5bc25e927eeaa6"} Jul 23 15:13:39.932594 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.932546 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-whcfg" event={"ID":"5e6d6e02-f435-4ccd-9b43-5c4f79d86b50","Type":"ContainerStarted","Data":"d3936b2e9685d31f24d559b2db74cda32eca9a2153257c1aadc2d0aee784277e"} Jul 23 15:13:39.932695 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.932671 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:39.945095 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.945041 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-s6czh" podStartSLOduration=42.096623294 podStartE2EDuration="55.945020713s" totalImagesPullingTime="13.848397419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:39.943988457 +0000 UTC m=+73.560035721" watchObservedRunningTime="2026-07-23 15:13:39.945020713 +0000 UTC m=+73.561067978" Jul 23 15:13:39.969421 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:39.969362 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-whcfg" podStartSLOduration=33.682563035 podStartE2EDuration="37.969345212s" totalImagesPullingTime="4.286782177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:39.968052276 +0000 UTC m=+73.584099556" watchObservedRunningTime="2026-07-23 15:13:39.969345212 +0000 UTC m=+73.585392476" Jul 23 15:13:40.017543 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:40.017496 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-v9g2q" podStartSLOduration=51.769861713 podStartE2EDuration="56.017479756s" totalImagesPullingTime="4.247618043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:40.016593162 +0000 UTC m=+73.632640427" watchObservedRunningTime="2026-07-23 15:13:40.017479756 +0000 UTC m=+73.633527019" Jul 23 15:13:40.049200 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:40.049149 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wl9qs" podStartSLOduration=67.110615128 podStartE2EDuration="1m12.049131887s" totalImagesPullingTime="4.938516759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:40.048173564 +0000 UTC m=+73.664220820" watchObservedRunningTime="2026-07-23 15:13:40.049131887 +0000 UTC m=+73.665179150" Jul 23 15:13:40.076677 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:40.074207 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-9mmv8" podStartSLOduration=51.657662513 podStartE2EDuration="56.074186756s" totalImagesPullingTime="4.416524243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:40.07216852 +0000 UTC m=+73.688215784" watchObservedRunningTime="2026-07-23 15:13:40.074186756 +0000 UTC m=+73.690234019" Jul 23 15:13:40.938040 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:40.938006 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" event={"ID":"aa8c3277-5337-4477-adf1-f2917985d495","Type":"ContainerStarted","Data":"580cb90e1f56c5015d7306f9c626421e9dd08e15499a38f230e033cb815a4a8f"} Jul 23 15:13:40.956458 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:40.956407 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9ljm6" podStartSLOduration=35.703131157 podStartE2EDuration="36.956388786s" totalImagesPullingTime="1.253257629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:13:40.95564267 +0000 UTC m=+74.571689934" watchObservedRunningTime="2026-07-23 15:13:40.956388786 +0000 UTC m=+74.572436049" Jul 23 15:13:49.941446 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:49.941323 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-whcfg" Jul 23 15:13:51.268362 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:51.268319 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:51.270688 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:51.270666 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9c43bfcd-1a27-4f40-a0c8-50e90817eb85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fwzgb\" (UID: \"9c43bfcd-1a27-4f40-a0c8-50e90817eb85\") " pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:51.524022 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:51.523935 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m6wjn\"" Jul 23 15:13:51.530374 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:51.530351 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fwzgb" Jul 23 15:13:51.721099 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:51.721063 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fwzgb"] Jul 23 15:13:51.976326 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:51.976277 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fwzgb" event={"ID":"9c43bfcd-1a27-4f40-a0c8-50e90817eb85","Type":"ContainerStarted","Data":"e429ec23e2aa6bd06becff98fc1e3a7baeeb0c7ab04775776a8451a6cc0afe1c"} Jul 23 15:13:54.434771 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:54.434726 2570 patch_prober.go:28] interesting pod/image-registry-758d7fd447-qkrz6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 15:13:54.435210 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:54.434798 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 15:13:55.899818 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:55.899785 2570 patch_prober.go:28] interesting pod/image-registry-758d7fd447-qkrz6 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 15:13:55.900175 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:55.899836 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 15:13:56.997527 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:13:56.997442 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fwzgb" event={"ID":"9c43bfcd-1a27-4f40-a0c8-50e90817eb85","Type":"ContainerStarted","Data":"14b49e2782cfae3aa596acef606278b3b6ba20051516552b78280a71430b6773"} Jul 23 15:14:00.012353 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:00.012310 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fwzgb" event={"ID":"9c43bfcd-1a27-4f40-a0c8-50e90817eb85","Type":"ContainerStarted","Data":"4b8556b1d83c4f1a98cd3fe4ffe92cd22d16ef1a83823b07884a864dd2d80470"} Jul 23 15:14:04.031008 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:04.030974 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fwzgb" event={"ID":"9c43bfcd-1a27-4f40-a0c8-50e90817eb85","Type":"ContainerStarted","Data":"6e33da1610f9eb838742ba2d01bee7e6700c30b10c7fd551fb134a0903303a33"} Jul 23 15:14:04.063803 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:04.063750 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fwzgb" podStartSLOduration=33.5896047 podStartE2EDuration="45.063734597s" totalImagesPullingTime="11.474129897s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:13:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:14:04.060382998 +0000 UTC m=+97.676430261" watchObservedRunningTime="2026-07-23 15:14:04.063734597 +0000 UTC m=+97.679781869" Jul 23 15:14:04.435080 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:04.435047 2570 patch_prober.go:28] interesting pod/image-registry-758d7fd447-qkrz6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 15:14:04.435247 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:04.435114 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 15:14:05.898534 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:05.898505 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ltmrf" Jul 23 15:14:05.900379 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:05.900354 2570 patch_prober.go:28] interesting pod/image-registry-758d7fd447-qkrz6 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 15:14:05.900503 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:05.900397 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 15:14:14.434091 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:14.434054 2570 patch_prober.go:28] interesting pod/image-registry-758d7fd447-qkrz6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 15:14:14.434470 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:14.434114 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 15:14:14.437730 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:14.437710 2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:14:14.438145 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:14.438112 2570 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"6c6ae522d9300332c062d56b123e57f873e5d5b46918554548bb8a4938dec3d3"} pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" containerMessage="Container registry failed liveness probe, will be restarted" Jul 23 15:14:14.441920 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:14.441890 2570 patch_prober.go:28] interesting pod/image-registry-758d7fd447-qkrz6 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 15:14:14.442053 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:14.441945 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 15:14:24.442115 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:24.442072 2570 patch_prober.go:28] interesting pod/image-registry-758d7fd447-qkrz6 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 15:14:24.442585 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:24.442126 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 15:14:29.114646 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.114611 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d5t78"] Jul 23 15:14:29.131111 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.131085 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.135577 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.135544 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 15:14:29.135961 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.135942 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 15:14:29.136074 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.135966 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 15:14:29.136126 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.136113 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fz765\"" Jul 23 15:14:29.144132 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.144115 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 15:14:29.290724 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.290691 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.290878 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.290736 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-textfile\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.290878 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.290789 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2jxx\" (UniqueName: \"kubernetes.io/projected/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-kube-api-access-t2jxx\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.290878 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.290828 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-sys\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.290878 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.290853 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-wtmp\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.291032 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.290885 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-metrics-client-ca\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.291032 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.290911 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-root\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.291032 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.290966 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-tls\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.291143 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.291031 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.392523 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.392432 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.392523 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.392473 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-textfile\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.392523 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.392498 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2jxx\" (UniqueName: \"kubernetes.io/projected/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-kube-api-access-t2jxx\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.392753 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.392537 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-sys\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.392753 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.392562 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-wtmp\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.392753 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.392599 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-metrics-client-ca\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.392753 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.392657 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-sys\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.392753 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.392713 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-root\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.392753 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.392742 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-tls\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.393028 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.392767 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-wtmp\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.393028 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.392779 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.393028 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.392836 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-root\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.393028 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.392904 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-textfile\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.393178 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.393137 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-accelerators-collector-config\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.393220 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.393178 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-metrics-client-ca\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.395053 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.395025 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.395143 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.395081 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-node-exporter-tls\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.401354 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.401329 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2jxx\" (UniqueName: \"kubernetes.io/projected/19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa-kube-api-access-t2jxx\") pod \"node-exporter-d5t78\" (UID: \"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa\") " pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.440745 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:29.440705 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d5t78" Jul 23 15:14:29.450580 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:14:29.450554 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d5a1c6_06e7_4f94_ae02_18f05b0bd9aa.slice/crio-ffd7c413566e8bea623c94279ed69fac96b20145d0f6b88c9043abdade76b87e WatchSource:0}: Error finding container ffd7c413566e8bea623c94279ed69fac96b20145d0f6b88c9043abdade76b87e: Status 404 returned error can't find the container with id ffd7c413566e8bea623c94279ed69fac96b20145d0f6b88c9043abdade76b87e Jul 23 15:14:30.135397 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:30.135360 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d5t78" event={"ID":"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa","Type":"ContainerStarted","Data":"ffd7c413566e8bea623c94279ed69fac96b20145d0f6b88c9043abdade76b87e"} Jul 23 15:14:31.140967 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:31.140927 2570 generic.go:356] "Generic (PLEG): container finished" podID="19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa" containerID="22f61b1386700da6d756fa21867ef9edb92570c705d337682930f5bbe2955c17" exitCode=0 Jul 23 15:14:31.141394 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:31.141009 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d5t78" event={"ID":"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa","Type":"ContainerDied","Data":"22f61b1386700da6d756fa21867ef9edb92570c705d337682930f5bbe2955c17"} Jul 23 15:14:32.150696 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:32.150665 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d5t78" event={"ID":"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa","Type":"ContainerStarted","Data":"0db0e1db5048a5acf743d2afc16303832822a3afbbaab8d078cf582139813ee6"} Jul 23 15:14:32.151128 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:32.150716 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d5t78" event={"ID":"19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa","Type":"ContainerStarted","Data":"0859a4f946519ecdc863c7c82019e265e99f77995a97e313d0f0f621a6610b99"} Jul 23 15:14:32.178687 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:32.178636 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d5t78" podStartSLOduration=2.127591021 podStartE2EDuration="3.17861885s" totalImagesPullingTime="988.010613ms" totalInitContainerRuntime="63.017216ms" isStatefulPod=true podCreationTimestamp="2026-07-23 15:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:14:32.177065556 +0000 UTC m=+125.793112820" watchObservedRunningTime="2026-07-23 15:14:32.17861885 +0000 UTC m=+125.794666113" Jul 23 15:14:34.446757 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:34.446733 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:14:37.865785 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:37.865749 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs\") pod \"network-metrics-daemon-n5624\" (UID: \"7b189e73-6f40-467e-9bf1-5505c5a7194c\") " pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:14:37.868059 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:37.868035 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b189e73-6f40-467e-9bf1-5505c5a7194c-metrics-certs\") pod \"network-metrics-daemon-n5624\" (UID: \"7b189e73-6f40-467e-9bf1-5505c5a7194c\") " pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:14:38.153501 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:38.153423 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7hjlh\"" Jul 23 15:14:38.161013 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:38.160995 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n5624" Jul 23 15:14:38.311194 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:38.311143 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n5624"] Jul 23 15:14:38.313393 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:14:38.313365 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b189e73_6f40_467e_9bf1_5505c5a7194c.slice/crio-9ab7c0771b86d351c38f993316a1323c6175a140531a1fb9c6d33dce8238abb0 WatchSource:0}: Error finding container 9ab7c0771b86d351c38f993316a1323c6175a140531a1fb9c6d33dce8238abb0: Status 404 returned error can't find the container with id 9ab7c0771b86d351c38f993316a1323c6175a140531a1fb9c6d33dce8238abb0 Jul 23 15:14:39.176557 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:39.176519 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n5624" event={"ID":"7b189e73-6f40-467e-9bf1-5505c5a7194c","Type":"ContainerStarted","Data":"9ab7c0771b86d351c38f993316a1323c6175a140531a1fb9c6d33dce8238abb0"} Jul 23 15:14:39.457421 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:39.457388 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" containerID="cri-o://6c6ae522d9300332c062d56b123e57f873e5d5b46918554548bb8a4938dec3d3" gracePeriod=30 Jul 23 15:14:40.182025 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:40.181988 2570 generic.go:356] "Generic (PLEG): container finished" podID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerID="6c6ae522d9300332c062d56b123e57f873e5d5b46918554548bb8a4938dec3d3" exitCode=0 Jul 23 15:14:40.182493 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:40.182065 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" event={"ID":"f9b322c9-ea12-44a0-994a-b481ef39e4e3","Type":"ContainerDied","Data":"6c6ae522d9300332c062d56b123e57f873e5d5b46918554548bb8a4938dec3d3"} Jul 23 15:14:40.182493 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:40.182141 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:14:40.182493 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:40.182161 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" event={"ID":"f9b322c9-ea12-44a0-994a-b481ef39e4e3","Type":"ContainerStarted","Data":"24b9ee78404195b8bbcba1bc933e5321260c827db1ee76a6a21c75df6ba1b113"} Jul 23 15:14:40.183885 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:40.183864 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n5624" event={"ID":"7b189e73-6f40-467e-9bf1-5505c5a7194c","Type":"ContainerStarted","Data":"618c929bbaf8444882952aa15b20c796b8d20fe925466df9b7040b2e5dc52ceb"} Jul 23 15:14:40.183967 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:40.183905 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n5624" event={"ID":"7b189e73-6f40-467e-9bf1-5505c5a7194c","Type":"ContainerStarted","Data":"e186f2e16c03f8c61cdd22030eb92117887d3405d15a3530887162cc7d551862"} Jul 23 15:14:40.238498 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:40.238447 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-n5624" podStartSLOduration=131.271713306 podStartE2EDuration="2m12.238428924s" totalImagesPullingTime="966.715618ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:14:40.236874335 +0000 UTC m=+133.852921603" watchObservedRunningTime="2026-07-23 15:14:40.238428924 +0000 UTC m=+133.854476188" Jul 23 15:14:41.926903 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:41.926865 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-758d7fd447-qkrz6"] Jul 23 15:14:44.203902 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:44.203872 2570 generic.go:356] "Generic (PLEG): container finished" podID="36dc10ed-f042-4994-8241-113f098485d6" containerID="5eed8e40ed0c72aed3853011093b7b0ae15538b5b8d94d04333f203ff93cd534" exitCode=0 Jul 23 15:14:44.204325 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:44.203926 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" event={"ID":"36dc10ed-f042-4994-8241-113f098485d6","Type":"ContainerDied","Data":"5eed8e40ed0c72aed3853011093b7b0ae15538b5b8d94d04333f203ff93cd534"} Jul 23 15:14:44.204325 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:44.204200 2570 scope.go:117] "RemoveContainer" containerID="5eed8e40ed0c72aed3853011093b7b0ae15538b5b8d94d04333f203ff93cd534" Jul 23 15:14:45.209042 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:45.209008 2570 generic.go:356] "Generic (PLEG): container finished" podID="71d3e351-b9cf-47fb-89b7-d69324c7a146" containerID="c4f31ec6c69b376e2f14d67a63e9d7c2cc9ad369f728867b3c4404547049102d" exitCode=0 Jul 23 15:14:45.209471 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:45.209085 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" event={"ID":"71d3e351-b9cf-47fb-89b7-d69324c7a146","Type":"ContainerDied","Data":"c4f31ec6c69b376e2f14d67a63e9d7c2cc9ad369f728867b3c4404547049102d"} Jul 23 15:14:45.209544 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:45.209533 2570 scope.go:117] "RemoveContainer" containerID="c4f31ec6c69b376e2f14d67a63e9d7c2cc9ad369f728867b3c4404547049102d" Jul 23 15:14:45.211072 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:45.211046 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-w5gjt" event={"ID":"36dc10ed-f042-4994-8241-113f098485d6","Type":"ContainerStarted","Data":"c73ffb7538e95087e7108f96583847ee56be03e4cf6c49087e200ac12af9b9d3"} Jul 23 15:14:46.216317 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:46.216266 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-2l9gg" event={"ID":"71d3e351-b9cf-47fb-89b7-d69324c7a146","Type":"ContainerStarted","Data":"6810bb6942126e22708efe3066d71e47e2c67a2701e02d1771bd30b798be5660"} Jul 23 15:14:52.699884 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:52.699843 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" podUID="fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 23 15:14:54.248185 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:54.248154 2570 generic.go:356] "Generic (PLEG): container finished" podID="48561ae8-8b4f-4367-a2ef-20584d90b8e4" containerID="d83d2bfde38da03309d09c3db6a69db93fcfc2f3d6128052046e0996deb1fcd4" exitCode=0 Jul 23 15:14:54.248578 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:54.248198 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-jhn9s" event={"ID":"48561ae8-8b4f-4367-a2ef-20584d90b8e4","Type":"ContainerDied","Data":"d83d2bfde38da03309d09c3db6a69db93fcfc2f3d6128052046e0996deb1fcd4"} Jul 23 15:14:54.248578 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:54.248529 2570 scope.go:117] "RemoveContainer" containerID="d83d2bfde38da03309d09c3db6a69db93fcfc2f3d6128052046e0996deb1fcd4" Jul 23 15:14:54.921949 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:54.921921 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74d5f4575d-4d249_f97ae7a3-b422-44a9-8942-752524470865/router/0.log" Jul 23 15:14:54.949418 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:54.949393 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8sl9b_e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9/serve-healthcheck-canary/0.log" Jul 23 15:14:55.254038 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:14:55.253943 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-jhn9s" event={"ID":"48561ae8-8b4f-4367-a2ef-20584d90b8e4","Type":"ContainerStarted","Data":"35560a1f3232ad302b01c439d00be1f13dc60612d8953cc5e2760b8a87e01bf1"} Jul 23 15:15:02.200194 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:02.200163 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:15:02.699418 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:02.699375 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" podUID="fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 23 15:15:07.214406 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.214361 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" containerID="cri-o://24b9ee78404195b8bbcba1bc933e5321260c827db1ee76a6a21c75df6ba1b113" gracePeriod=30 Jul 23 15:15:07.487074 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.487049 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:15:07.613315 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.613269 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f9b322c9-ea12-44a0-994a-b481ef39e4e3-trusted-ca\") pod \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " Jul 23 15:15:07.613473 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.613344 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-bound-sa-token\") pod \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " Jul 23 15:15:07.613473 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.613367 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-certificates\") pod \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " Jul 23 15:15:07.613473 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.613407 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klgpz\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-kube-api-access-klgpz\") pod \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " Jul 23 15:15:07.613473 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.613451 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f9b322c9-ea12-44a0-994a-b481ef39e4e3-image-registry-private-configuration\") pod \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " Jul 23 15:15:07.613679 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.613519 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f9b322c9-ea12-44a0-994a-b481ef39e4e3-installation-pull-secrets\") pod \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " Jul 23 15:15:07.613679 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.613570 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls\") pod \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " Jul 23 15:15:07.613784 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.613696 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9b322c9-ea12-44a0-994a-b481ef39e4e3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f9b322c9-ea12-44a0-994a-b481ef39e4e3" (UID: "f9b322c9-ea12-44a0-994a-b481ef39e4e3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:15:07.613784 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.613748 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f9b322c9-ea12-44a0-994a-b481ef39e4e3" (UID: "f9b322c9-ea12-44a0-994a-b481ef39e4e3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:15:07.613896 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.613795 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f9b322c9-ea12-44a0-994a-b481ef39e4e3-ca-trust-extracted\") pod \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\" (UID: \"f9b322c9-ea12-44a0-994a-b481ef39e4e3\") " Jul 23 15:15:07.614110 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.614061 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f9b322c9-ea12-44a0-994a-b481ef39e4e3-trusted-ca\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:15:07.614110 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.614086 2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-certificates\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:15:07.616348 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.616311 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9b322c9-ea12-44a0-994a-b481ef39e4e3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f9b322c9-ea12-44a0-994a-b481ef39e4e3" (UID: "f9b322c9-ea12-44a0-994a-b481ef39e4e3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:15:07.616447 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.616399 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9b322c9-ea12-44a0-994a-b481ef39e4e3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f9b322c9-ea12-44a0-994a-b481ef39e4e3" (UID: "f9b322c9-ea12-44a0-994a-b481ef39e4e3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:15:07.616505 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.616450 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-kube-api-access-klgpz" (OuterVolumeSpecName: "kube-api-access-klgpz") pod "f9b322c9-ea12-44a0-994a-b481ef39e4e3" (UID: "f9b322c9-ea12-44a0-994a-b481ef39e4e3"). InnerVolumeSpecName "kube-api-access-klgpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:15:07.616543 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.616526 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f9b322c9-ea12-44a0-994a-b481ef39e4e3" (UID: "f9b322c9-ea12-44a0-994a-b481ef39e4e3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:15:07.617585 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.617563 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f9b322c9-ea12-44a0-994a-b481ef39e4e3" (UID: "f9b322c9-ea12-44a0-994a-b481ef39e4e3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:15:07.624008 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.623980 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b322c9-ea12-44a0-994a-b481ef39e4e3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f9b322c9-ea12-44a0-994a-b481ef39e4e3" (UID: "f9b322c9-ea12-44a0-994a-b481ef39e4e3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:15:07.714715 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.714682 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klgpz\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-kube-api-access-klgpz\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:15:07.714715 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.714715 2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f9b322c9-ea12-44a0-994a-b481ef39e4e3-image-registry-private-configuration\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:15:07.714911 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.714729 2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f9b322c9-ea12-44a0-994a-b481ef39e4e3-installation-pull-secrets\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:15:07.714911 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.714743 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-registry-tls\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:15:07.714911 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.714755 2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f9b322c9-ea12-44a0-994a-b481ef39e4e3-ca-trust-extracted\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:15:07.714911 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:07.714765 2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9b322c9-ea12-44a0-994a-b481ef39e4e3-bound-sa-token\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:15:08.313502 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:08.313455 2570 generic.go:356] "Generic (PLEG): container finished" podID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerID="24b9ee78404195b8bbcba1bc933e5321260c827db1ee76a6a21c75df6ba1b113" exitCode=0 Jul 23 15:15:08.313979 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:08.313611 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" event={"ID":"f9b322c9-ea12-44a0-994a-b481ef39e4e3","Type":"ContainerDied","Data":"24b9ee78404195b8bbcba1bc933e5321260c827db1ee76a6a21c75df6ba1b113"} Jul 23 15:15:08.313979 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:08.313710 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" event={"ID":"f9b322c9-ea12-44a0-994a-b481ef39e4e3","Type":"ContainerDied","Data":"1ea0815998a05c3c30cc55ef2aa3472f57c3f10fd268768df413c4921b9dc875"} Jul 23 15:15:08.313979 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:08.313745 2570 scope.go:117] "RemoveContainer" containerID="24b9ee78404195b8bbcba1bc933e5321260c827db1ee76a6a21c75df6ba1b113" Jul 23 15:15:08.313979 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:08.313913 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758d7fd447-qkrz6" Jul 23 15:15:08.330802 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:08.330774 2570 scope.go:117] "RemoveContainer" containerID="6c6ae522d9300332c062d56b123e57f873e5d5b46918554548bb8a4938dec3d3" Jul 23 15:15:08.338911 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:08.338892 2570 scope.go:117] "RemoveContainer" containerID="24b9ee78404195b8bbcba1bc933e5321260c827db1ee76a6a21c75df6ba1b113" Jul 23 15:15:08.339216 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:15:08.339195 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24b9ee78404195b8bbcba1bc933e5321260c827db1ee76a6a21c75df6ba1b113\": container with ID starting with 24b9ee78404195b8bbcba1bc933e5321260c827db1ee76a6a21c75df6ba1b113 not found: ID does not exist" containerID="24b9ee78404195b8bbcba1bc933e5321260c827db1ee76a6a21c75df6ba1b113" Jul 23 15:15:08.339268 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:08.339227 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24b9ee78404195b8bbcba1bc933e5321260c827db1ee76a6a21c75df6ba1b113"} err="failed to get container status \"24b9ee78404195b8bbcba1bc933e5321260c827db1ee76a6a21c75df6ba1b113\": rpc error: code = NotFound desc = could not find container \"24b9ee78404195b8bbcba1bc933e5321260c827db1ee76a6a21c75df6ba1b113\": container with ID starting with 24b9ee78404195b8bbcba1bc933e5321260c827db1ee76a6a21c75df6ba1b113 not found: ID does not exist" Jul 23 15:15:08.339268 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:08.339246 2570 scope.go:117] "RemoveContainer" containerID="6c6ae522d9300332c062d56b123e57f873e5d5b46918554548bb8a4938dec3d3" Jul 23 15:15:08.339507 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:15:08.339491 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c6ae522d9300332c062d56b123e57f873e5d5b46918554548bb8a4938dec3d3\": container with ID starting with 6c6ae522d9300332c062d56b123e57f873e5d5b46918554548bb8a4938dec3d3 not found: ID does not exist" containerID="6c6ae522d9300332c062d56b123e57f873e5d5b46918554548bb8a4938dec3d3" Jul 23 15:15:08.339546 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:08.339514 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c6ae522d9300332c062d56b123e57f873e5d5b46918554548bb8a4938dec3d3"} err="failed to get container status \"6c6ae522d9300332c062d56b123e57f873e5d5b46918554548bb8a4938dec3d3\": rpc error: code = NotFound desc = could not find container \"6c6ae522d9300332c062d56b123e57f873e5d5b46918554548bb8a4938dec3d3\": container with ID starting with 6c6ae522d9300332c062d56b123e57f873e5d5b46918554548bb8a4938dec3d3 not found: ID does not exist" Jul 23 15:15:08.342738 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:08.342716 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-758d7fd447-qkrz6"] Jul 23 15:15:08.346881 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:08.346858 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-758d7fd447-qkrz6"] Jul 23 15:15:08.438697 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:08.438658 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" path="/var/lib/kubelet/pods/f9b322c9-ea12-44a0-994a-b481ef39e4e3/volumes" Jul 23 15:15:12.698992 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:12.698954 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" podUID="fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 23 15:15:12.699474 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:12.699024 2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" Jul 23 15:15:12.699522 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:12.699509 2570 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"2ea5b3e1cb89456770613115e008d95bf6cc54730e3f60c3f67279b987d3559d"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 23 15:15:12.699563 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:12.699544 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" podUID="fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32" containerName="service-proxy" containerID="cri-o://2ea5b3e1cb89456770613115e008d95bf6cc54730e3f60c3f67279b987d3559d" gracePeriod=30 Jul 23 15:15:13.336245 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:13.336211 2570 generic.go:356] "Generic (PLEG): container finished" podID="fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32" containerID="2ea5b3e1cb89456770613115e008d95bf6cc54730e3f60c3f67279b987d3559d" exitCode=2 Jul 23 15:15:13.336435 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:13.336285 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" event={"ID":"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32","Type":"ContainerDied","Data":"2ea5b3e1cb89456770613115e008d95bf6cc54730e3f60c3f67279b987d3559d"} Jul 23 15:15:13.336435 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:15:13.336354 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86cf8c7cd5-6hrgl" event={"ID":"fad7abcf-b2ae-4d6f-81c6-f1dad9f79e32","Type":"ContainerStarted","Data":"6ddd737fd3c5d0fc0197aeea03be66207822d32ecb5d3bcf949fc51cf5242433"} Jul 23 15:17:28.017810 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:17:28.017772 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s6czh_5522141f-9f92-48ad-a954-7c0bff719ebd/console-operator/1.log" Jul 23 15:17:28.021514 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:17:28.021490 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cdjsd_946dd7b0-9ff6-41fe-9515-3e1d5b2e3867/ovn-acl-logging/0.log" Jul 23 15:17:28.024582 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:17:28.024564 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s6czh_5522141f-9f92-48ad-a954-7c0bff719ebd/console-operator/1.log" Jul 23 15:17:28.028382 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:17:28.028364 2570 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 15:17:28.028512 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:17:28.028498 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cdjsd_946dd7b0-9ff6-41fe-9515-3e1d5b2e3867/ovn-acl-logging/0.log" Jul 23 15:22:28.041944 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:28.041918 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s6czh_5522141f-9f92-48ad-a954-7c0bff719ebd/console-operator/1.log" Jul 23 15:22:28.046087 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:28.046064 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cdjsd_946dd7b0-9ff6-41fe-9515-3e1d5b2e3867/ovn-acl-logging/0.log" Jul 23 15:22:28.050988 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:28.050970 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s6czh_5522141f-9f92-48ad-a954-7c0bff719ebd/console-operator/1.log" Jul 23 15:22:28.054660 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:28.054643 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cdjsd_946dd7b0-9ff6-41fe-9515-3e1d5b2e3867/ovn-acl-logging/0.log" Jul 23 15:22:37.677358 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.677321 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc"] Jul 23 15:22:37.677739 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.677583 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" Jul 23 15:22:37.677739 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.677593 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" Jul 23 15:22:37.677739 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.677653 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" Jul 23 15:22:37.677739 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.677662 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" Jul 23 15:22:37.677739 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.677703 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" Jul 23 15:22:37.677739 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.677709 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9b322c9-ea12-44a0-994a-b481ef39e4e3" containerName="registry" Jul 23 15:22:37.680447 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.680430 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.685852 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.685828 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 15:22:37.685992 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.685832 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 15:22:37.696380 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.696360 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 15:22:37.696724 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.696708 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-g8kmd\"" Jul 23 15:22:37.696884 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.696872 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 15:22:37.710450 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.710429 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc"] Jul 23 15:22:37.807925 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.807892 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b532a34f-49c8-4b1f-b741-155d83d5215e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.808097 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.807940 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b532a34f-49c8-4b1f-b741-155d83d5215e-manifests\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.808097 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.807971 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b532a34f-49c8-4b1f-b741-155d83d5215e-webhook-cert\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.808097 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.808004 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj4gk\" (UniqueName: \"kubernetes.io/projected/b532a34f-49c8-4b1f-b741-155d83d5215e-kube-api-access-bj4gk\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.808097 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.808043 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b532a34f-49c8-4b1f-b741-155d83d5215e-tmp\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.909212 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.909173 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b532a34f-49c8-4b1f-b741-155d83d5215e-tmp\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.909419 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.909228 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b532a34f-49c8-4b1f-b741-155d83d5215e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.909419 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.909261 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b532a34f-49c8-4b1f-b741-155d83d5215e-manifests\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.909419 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.909281 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b532a34f-49c8-4b1f-b741-155d83d5215e-webhook-cert\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.909419 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.909342 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj4gk\" (UniqueName: \"kubernetes.io/projected/b532a34f-49c8-4b1f-b741-155d83d5215e-kube-api-access-bj4gk\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.909747 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.909720 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b532a34f-49c8-4b1f-b741-155d83d5215e-manifests\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.911660 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.911627 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b532a34f-49c8-4b1f-b741-155d83d5215e-tmp\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.911762 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.911711 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b532a34f-49c8-4b1f-b741-155d83d5215e-webhook-cert\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.911854 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.911837 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b532a34f-49c8-4b1f-b741-155d83d5215e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.921497 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.921474 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj4gk\" (UniqueName: \"kubernetes.io/projected/b532a34f-49c8-4b1f-b741-155d83d5215e-kube-api-access-bj4gk\") pod \"opendatahub-operator-controller-manager-777bc59866-mbqvc\" (UID: \"b532a34f-49c8-4b1f-b741-155d83d5215e\") " pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:37.990859 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:37.990780 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:38.168429 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:38.168397 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc"] Jul 23 15:22:38.171658 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:22:38.171629 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb532a34f_49c8_4b1f_b741_155d83d5215e.slice/crio-4e6c123d163b40651ad5fa2094b5e026c47a66fa1d5bc01b5b7ad835c967213b WatchSource:0}: Error finding container 4e6c123d163b40651ad5fa2094b5e026c47a66fa1d5bc01b5b7ad835c967213b: Status 404 returned error can't find the container with id 4e6c123d163b40651ad5fa2094b5e026c47a66fa1d5bc01b5b7ad835c967213b Jul 23 15:22:38.173348 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:38.173333 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 15:22:38.977438 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:38.977392 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" event={"ID":"b532a34f-49c8-4b1f-b741-155d83d5215e","Type":"ContainerStarted","Data":"4e6c123d163b40651ad5fa2094b5e026c47a66fa1d5bc01b5b7ad835c967213b"} Jul 23 15:22:41.992526 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:41.992493 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" event={"ID":"b532a34f-49c8-4b1f-b741-155d83d5215e","Type":"ContainerStarted","Data":"2117ed1c9cccabc846e2780b55b2f8188831d25441ed4bc5b9717c79351b8d49"} Jul 23 15:22:42.998215 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:42.998177 2570 generic.go:356] "Generic (PLEG): container finished" podID="b532a34f-49c8-4b1f-b741-155d83d5215e" containerID="2117ed1c9cccabc846e2780b55b2f8188831d25441ed4bc5b9717c79351b8d49" exitCode=0 Jul 23 15:22:42.998718 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:42.998225 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" event={"ID":"b532a34f-49c8-4b1f-b741-155d83d5215e","Type":"ContainerDied","Data":"2117ed1c9cccabc846e2780b55b2f8188831d25441ed4bc5b9717c79351b8d49"} Jul 23 15:22:44.004135 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:44.004097 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" event={"ID":"b532a34f-49c8-4b1f-b741-155d83d5215e","Type":"ContainerStarted","Data":"6cdae50e476aed8e8ccd0f15a0a07b92ee74eabddb98f548760ed5da56765d66"} Jul 23 15:22:44.004634 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:44.004205 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:55.011138 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:55.011063 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" Jul 23 15:22:55.067691 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:55.067637 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-777bc59866-mbqvc" podStartSLOduration=13.684649121 podStartE2EDuration="18.067614053s" totalImagesPullingTime="3.339047141s" totalInitContainerRuntime="1.043917791s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:22:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:22:44.045264574 +0000 UTC m=+617.661311837" watchObservedRunningTime="2026-07-23 15:22:55.067614053 +0000 UTC m=+628.683661318" Jul 23 15:22:57.926792 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:57.926762 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2"] Jul 23 15:22:57.933101 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:57.933079 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2" Jul 23 15:22:57.936589 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:57.936571 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 23 15:22:57.936715 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:57.936571 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 23 15:22:57.936806 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:57.936794 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wn2bp\"" Jul 23 15:22:57.967120 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:57.967098 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2"] Jul 23 15:22:58.073046 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.073019 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzxct\" (UniqueName: \"kubernetes.io/projected/3ff19238-e3df-4b67-8819-167a11efeb1d-kube-api-access-nzxct\") pod \"kube-auth-proxy-77f87dcdfb-sg7b2\" (UID: \"3ff19238-e3df-4b67-8819-167a11efeb1d\") " pod="openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2" Jul 23 15:22:58.073233 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.073061 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ff19238-e3df-4b67-8819-167a11efeb1d-tls-certs\") pod \"kube-auth-proxy-77f87dcdfb-sg7b2\" (UID: \"3ff19238-e3df-4b67-8819-167a11efeb1d\") " pod="openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2" Jul 23 15:22:58.073233 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.073098 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ff19238-e3df-4b67-8819-167a11efeb1d-tmp\") pod \"kube-auth-proxy-77f87dcdfb-sg7b2\" (UID: \"3ff19238-e3df-4b67-8819-167a11efeb1d\") " pod="openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2" Jul 23 15:22:58.173519 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.173489 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ff19238-e3df-4b67-8819-167a11efeb1d-tmp\") pod \"kube-auth-proxy-77f87dcdfb-sg7b2\" (UID: \"3ff19238-e3df-4b67-8819-167a11efeb1d\") " pod="openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2" Jul 23 15:22:58.173697 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.173543 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzxct\" (UniqueName: \"kubernetes.io/projected/3ff19238-e3df-4b67-8819-167a11efeb1d-kube-api-access-nzxct\") pod \"kube-auth-proxy-77f87dcdfb-sg7b2\" (UID: \"3ff19238-e3df-4b67-8819-167a11efeb1d\") " pod="openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2" Jul 23 15:22:58.173697 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.173575 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ff19238-e3df-4b67-8819-167a11efeb1d-tls-certs\") pod \"kube-auth-proxy-77f87dcdfb-sg7b2\" (UID: \"3ff19238-e3df-4b67-8819-167a11efeb1d\") " pod="openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2" Jul 23 15:22:58.175860 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.175831 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ff19238-e3df-4b67-8819-167a11efeb1d-tmp\") pod \"kube-auth-proxy-77f87dcdfb-sg7b2\" (UID: \"3ff19238-e3df-4b67-8819-167a11efeb1d\") " pod="openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2" Jul 23 15:22:58.176065 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.176046 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ff19238-e3df-4b67-8819-167a11efeb1d-tls-certs\") pod \"kube-auth-proxy-77f87dcdfb-sg7b2\" (UID: \"3ff19238-e3df-4b67-8819-167a11efeb1d\") " pod="openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2" Jul 23 15:22:58.178171 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.178109 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kdk4x"] Jul 23 15:22:58.180364 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.180346 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kdk4x" Jul 23 15:22:58.183130 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.183108 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 23 15:22:58.183222 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.183108 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-c82j4\"" Jul 23 15:22:58.199348 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.199328 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kdk4x"] Jul 23 15:22:58.201171 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.201146 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzxct\" (UniqueName: \"kubernetes.io/projected/3ff19238-e3df-4b67-8819-167a11efeb1d-kube-api-access-nzxct\") pod \"kube-auth-proxy-77f87dcdfb-sg7b2\" (UID: \"3ff19238-e3df-4b67-8819-167a11efeb1d\") " pod="openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2" Jul 23 15:22:58.241625 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.241596 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2" Jul 23 15:22:58.275045 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.274990 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sstp\" (UniqueName: \"kubernetes.io/projected/a8b33059-f17d-4231-a25a-ef79efd07c39-kube-api-access-2sstp\") pod \"odh-model-controller-7998c67dc4-kdk4x\" (UID: \"a8b33059-f17d-4231-a25a-ef79efd07c39\") " pod="opendatahub/odh-model-controller-7998c67dc4-kdk4x" Jul 23 15:22:58.275177 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.275053 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8b33059-f17d-4231-a25a-ef79efd07c39-cert\") pod \"odh-model-controller-7998c67dc4-kdk4x\" (UID: \"a8b33059-f17d-4231-a25a-ef79efd07c39\") " pod="opendatahub/odh-model-controller-7998c67dc4-kdk4x" Jul 23 15:22:58.375817 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.375783 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sstp\" (UniqueName: \"kubernetes.io/projected/a8b33059-f17d-4231-a25a-ef79efd07c39-kube-api-access-2sstp\") pod \"odh-model-controller-7998c67dc4-kdk4x\" (UID: \"a8b33059-f17d-4231-a25a-ef79efd07c39\") " pod="opendatahub/odh-model-controller-7998c67dc4-kdk4x" Jul 23 15:22:58.375817 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.375823 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8b33059-f17d-4231-a25a-ef79efd07c39-cert\") pod \"odh-model-controller-7998c67dc4-kdk4x\" (UID: \"a8b33059-f17d-4231-a25a-ef79efd07c39\") " pod="opendatahub/odh-model-controller-7998c67dc4-kdk4x" Jul 23 15:22:58.376002 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:22:58.375956 2570 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 23 15:22:58.376037 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:22:58.376018 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8b33059-f17d-4231-a25a-ef79efd07c39-cert podName:a8b33059-f17d-4231-a25a-ef79efd07c39 nodeName:}" failed. No retries permitted until 2026-07-23 15:22:58.876001632 +0000 UTC m=+632.492048874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a8b33059-f17d-4231-a25a-ef79efd07c39-cert") pod "odh-model-controller-7998c67dc4-kdk4x" (UID: "a8b33059-f17d-4231-a25a-ef79efd07c39") : secret "odh-model-controller-webhook-cert" not found Jul 23 15:22:58.384584 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.384555 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2"] Jul 23 15:22:58.389494 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:22:58.389472 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff19238_e3df_4b67_8819_167a11efeb1d.slice/crio-1d5f370426c814cd7733b71dc5e1f6b7387c7247a154d62d81f5c247a4c5db1a WatchSource:0}: Error finding container 1d5f370426c814cd7733b71dc5e1f6b7387c7247a154d62d81f5c247a4c5db1a: Status 404 returned error can't find the container with id 1d5f370426c814cd7733b71dc5e1f6b7387c7247a154d62d81f5c247a4c5db1a Jul 23 15:22:58.408053 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.408026 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sstp\" (UniqueName: \"kubernetes.io/projected/a8b33059-f17d-4231-a25a-ef79efd07c39-kube-api-access-2sstp\") pod \"odh-model-controller-7998c67dc4-kdk4x\" (UID: \"a8b33059-f17d-4231-a25a-ef79efd07c39\") " pod="opendatahub/odh-model-controller-7998c67dc4-kdk4x" Jul 23 15:22:58.881079 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.881039 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8b33059-f17d-4231-a25a-ef79efd07c39-cert\") pod \"odh-model-controller-7998c67dc4-kdk4x\" (UID: \"a8b33059-f17d-4231-a25a-ef79efd07c39\") " pod="opendatahub/odh-model-controller-7998c67dc4-kdk4x" Jul 23 15:22:58.883477 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:58.883456 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8b33059-f17d-4231-a25a-ef79efd07c39-cert\") pod \"odh-model-controller-7998c67dc4-kdk4x\" (UID: \"a8b33059-f17d-4231-a25a-ef79efd07c39\") " pod="opendatahub/odh-model-controller-7998c67dc4-kdk4x" Jul 23 15:22:59.064632 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.064596 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2" event={"ID":"3ff19238-e3df-4b67-8819-167a11efeb1d","Type":"ContainerStarted","Data":"1d5f370426c814cd7733b71dc5e1f6b7387c7247a154d62d81f5c247a4c5db1a"} Jul 23 15:22:59.096197 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.096166 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kdk4x" Jul 23 15:22:59.239233 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.239202 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7hg42"] Jul 23 15:22:59.241696 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.241673 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7hg42" Jul 23 15:22:59.244469 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.244444 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 23 15:22:59.254261 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.254222 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kdk4x"] Jul 23 15:22:59.256006 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.255973 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7hg42"] Jul 23 15:22:59.384647 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.384615 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b820a833-0c40-47e7-a675-faf0f3eacd95-tls-certs\") pod \"model-serving-api-54d89bd5fc-7hg42\" (UID: \"b820a833-0c40-47e7-a675-faf0f3eacd95\") " pod="opendatahub/model-serving-api-54d89bd5fc-7hg42" Jul 23 15:22:59.384813 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.384721 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg6xs\" (UniqueName: \"kubernetes.io/projected/b820a833-0c40-47e7-a675-faf0f3eacd95-kube-api-access-kg6xs\") pod \"model-serving-api-54d89bd5fc-7hg42\" (UID: \"b820a833-0c40-47e7-a675-faf0f3eacd95\") " pod="opendatahub/model-serving-api-54d89bd5fc-7hg42" Jul 23 15:22:59.485687 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.485612 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b820a833-0c40-47e7-a675-faf0f3eacd95-tls-certs\") pod \"model-serving-api-54d89bd5fc-7hg42\" (UID: \"b820a833-0c40-47e7-a675-faf0f3eacd95\") " pod="opendatahub/model-serving-api-54d89bd5fc-7hg42" Jul 23 15:22:59.485842 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.485693 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg6xs\" (UniqueName: \"kubernetes.io/projected/b820a833-0c40-47e7-a675-faf0f3eacd95-kube-api-access-kg6xs\") pod \"model-serving-api-54d89bd5fc-7hg42\" (UID: \"b820a833-0c40-47e7-a675-faf0f3eacd95\") " pod="opendatahub/model-serving-api-54d89bd5fc-7hg42" Jul 23 15:22:59.488462 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.488433 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b820a833-0c40-47e7-a675-faf0f3eacd95-tls-certs\") pod \"model-serving-api-54d89bd5fc-7hg42\" (UID: \"b820a833-0c40-47e7-a675-faf0f3eacd95\") " pod="opendatahub/model-serving-api-54d89bd5fc-7hg42" Jul 23 15:22:59.495757 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.495734 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg6xs\" (UniqueName: \"kubernetes.io/projected/b820a833-0c40-47e7-a675-faf0f3eacd95-kube-api-access-kg6xs\") pod \"model-serving-api-54d89bd5fc-7hg42\" (UID: \"b820a833-0c40-47e7-a675-faf0f3eacd95\") " pod="opendatahub/model-serving-api-54d89bd5fc-7hg42" Jul 23 15:22:59.529444 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.529406 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7"] Jul 23 15:22:59.531794 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.531774 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.534424 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.534398 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 15:22:59.534547 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.534400 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 15:22:59.534611 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.534546 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 15:22:59.534692 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.534676 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-74kj4\"" Jul 23 15:22:59.534925 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.534910 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 15:22:59.548871 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.548846 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7"] Jul 23 15:22:59.555918 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.555899 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7hg42" Jul 23 15:22:59.687423 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.687394 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e5d575a4-e703-47d7-a632-b65328748130-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.687591 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.687433 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5d575a4-e703-47d7-a632-b65328748130-local-certs\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.687591 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.687452 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2tzx\" (UniqueName: \"kubernetes.io/projected/e5d575a4-e703-47d7-a632-b65328748130-kube-api-access-t2tzx\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.687591 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.687500 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.687591 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.687563 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.687720 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.687612 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5d575a4-e703-47d7-a632-b65328748130-istio-token\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.687720 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.687645 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-cacerts\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.706878 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.706849 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7hg42"] Jul 23 15:22:59.709740 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:22:59.709716 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb820a833_0c40_47e7_a675_faf0f3eacd95.slice/crio-9a4cfeb4e5f2aa2bece135aa3a170c1ddb4be6334bd2db9d96540dbce456a80d WatchSource:0}: Error finding container 9a4cfeb4e5f2aa2bece135aa3a170c1ddb4be6334bd2db9d96540dbce456a80d: Status 404 returned error can't find the container with id 9a4cfeb4e5f2aa2bece135aa3a170c1ddb4be6334bd2db9d96540dbce456a80d Jul 23 15:22:59.788966 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.788883 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.788966 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.788950 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5d575a4-e703-47d7-a632-b65328748130-istio-token\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.789192 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.788978 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-cacerts\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.789192 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.789012 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e5d575a4-e703-47d7-a632-b65328748130-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.789192 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.789046 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5d575a4-e703-47d7-a632-b65328748130-local-certs\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.789192 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.789073 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2tzx\" (UniqueName: \"kubernetes.io/projected/e5d575a4-e703-47d7-a632-b65328748130-kube-api-access-t2tzx\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.789192 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.789105 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.789977 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.789919 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e5d575a4-e703-47d7-a632-b65328748130-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.791890 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.791869 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5d575a4-e703-47d7-a632-b65328748130-local-certs\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.792078 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.792057 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.792153 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.792062 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.792153 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.792108 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-cacerts\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.804791 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.804769 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5d575a4-e703-47d7-a632-b65328748130-istio-token\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.807627 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.807608 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2tzx\" (UniqueName: \"kubernetes.io/projected/e5d575a4-e703-47d7-a632-b65328748130-kube-api-access-t2tzx\") pod \"istiod-openshift-gateway-7574459656-jnbp7\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:22:59.843460 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:22:59.843432 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:23:00.004803 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:00.004772 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7"] Jul 23 15:23:00.008012 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:23:00.007979 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d575a4_e703_47d7_a632_b65328748130.slice/crio-98279adadf0420575e87ebe9987ba04d2fd0c3e054b19b4ef5e3e9ad439b958c WatchSource:0}: Error finding container 98279adadf0420575e87ebe9987ba04d2fd0c3e054b19b4ef5e3e9ad439b958c: Status 404 returned error can't find the container with id 98279adadf0420575e87ebe9987ba04d2fd0c3e054b19b4ef5e3e9ad439b958c Jul 23 15:23:00.070736 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:00.070645 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kdk4x" event={"ID":"a8b33059-f17d-4231-a25a-ef79efd07c39","Type":"ContainerStarted","Data":"58838f6ac0203dc426b4eb839e90bdcd0c5441aa4b8eeb4dd883861ea76bb940"} Jul 23 15:23:00.071937 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:00.071907 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" event={"ID":"e5d575a4-e703-47d7-a632-b65328748130","Type":"ContainerStarted","Data":"98279adadf0420575e87ebe9987ba04d2fd0c3e054b19b4ef5e3e9ad439b958c"} Jul 23 15:23:00.073178 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:00.073153 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7hg42" event={"ID":"b820a833-0c40-47e7-a675-faf0f3eacd95","Type":"ContainerStarted","Data":"9a4cfeb4e5f2aa2bece135aa3a170c1ddb4be6334bd2db9d96540dbce456a80d"} Jul 23 15:23:04.044643 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:04.044598 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:23:04.045004 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:04.044685 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:23:05.101280 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:05.101243 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kdk4x" event={"ID":"a8b33059-f17d-4231-a25a-ef79efd07c39","Type":"ContainerStarted","Data":"a6f45591dc5445847eff920cd6f020c1aa82e27ff42f2fabc8219d4a55f72e5e"} Jul 23 15:23:05.101780 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:05.101331 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-kdk4x" Jul 23 15:23:05.103230 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:05.103198 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" event={"ID":"e5d575a4-e703-47d7-a632-b65328748130","Type":"ContainerStarted","Data":"046b3c9b287142a5ff5c5d1e2c4dc95415aaccf9eaed6f2492863063545f8d49"} Jul 23 15:23:05.103392 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:05.103253 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:23:05.104979 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:05.104927 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7hg42" event={"ID":"b820a833-0c40-47e7-a675-faf0f3eacd95","Type":"ContainerStarted","Data":"480a77bdc0f20a3c72343e8084d71d6c3cd7eeeecf6a70da9cfc9e05601fb6b7"} Jul 23 15:23:05.105098 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:05.105078 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-7hg42" Jul 23 15:23:05.105507 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:05.105485 2570 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-jnbp7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 15:23:05.105909 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:05.105522 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" podUID="e5d575a4-e703-47d7-a632-b65328748130" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 15:23:05.106664 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:05.106632 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2" event={"ID":"3ff19238-e3df-4b67-8819-167a11efeb1d","Type":"ContainerStarted","Data":"aa8d6b9e00a45a8679e2112d52b3d872c18e01227f45e446d4f9a6c3f40018ab"} Jul 23 15:23:05.215057 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:05.214996 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77f87dcdfb-sg7b2" podStartSLOduration=3.454087883 podStartE2EDuration="8.214973538s" totalImagesPullingTime="4.760885655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:22:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:23:05.213669293 +0000 UTC m=+638.829716559" watchObservedRunningTime="2026-07-23 15:23:05.214973538 +0000 UTC m=+638.831020802" Jul 23 15:23:05.215495 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:05.215469 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-kdk4x" podStartSLOduration=2.440080537 podStartE2EDuration="7.215461014s" totalImagesPullingTime="4.775380477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:22:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:23:05.153439926 +0000 UTC m=+638.769487190" watchObservedRunningTime="2026-07-23 15:23:05.215461014 +0000 UTC m=+638.831508277" Jul 23 15:23:05.256120 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:05.256065 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" podStartSLOduration=2.223239855 podStartE2EDuration="6.256044073s" totalImagesPullingTime="4.032804218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:23:05.255460666 +0000 UTC m=+638.871507932" watchObservedRunningTime="2026-07-23 15:23:05.256044073 +0000 UTC m=+638.872091314" Jul 23 15:23:05.297418 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:05.297354 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-7hg42" podStartSLOduration=2.011074329 podStartE2EDuration="6.297333577s" totalImagesPullingTime="4.286259248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:23:05.297127021 +0000 UTC m=+638.913174286" watchObservedRunningTime="2026-07-23 15:23:05.297333577 +0000 UTC m=+638.913380842" Jul 23 15:23:06.111720 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.111693 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:23:06.263730 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.263698 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q"] Jul 23 15:23:06.267120 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.267099 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.270403 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.270370 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fccfs\"" Jul 23 15:23:06.279682 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.279650 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q"] Jul 23 15:23:06.351236 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.351204 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.351417 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.351254 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.351417 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.351278 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.351417 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.351381 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.351542 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.351422 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.351542 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.351499 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.351542 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.351520 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.351641 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.351559 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.351641 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.351577 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wptvg\" (UniqueName: \"kubernetes.io/projected/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-kube-api-access-wptvg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.452060 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.451974 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.452060 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.452024 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.452060 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.452049 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.452351 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.452076 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.452351 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.452126 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.452351 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.452141 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.452351 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.452174 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.452351 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.452192 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wptvg\" (UniqueName: \"kubernetes.io/projected/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-kube-api-access-wptvg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.452351 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.452227 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.452699 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.452630 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.452699 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.452657 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.452781 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.452727 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.452781 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.452734 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.453170 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.453148 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.454499 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.454479 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.454643 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.454625 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.487530 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.487505 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.489929 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.489908 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wptvg\" (UniqueName: \"kubernetes.io/projected/0b22cc29-2cb5-442c-a14f-84c0992bd6c9-kube-api-access-wptvg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4p6r6q\" (UID: \"0b22cc29-2cb5-442c-a14f-84c0992bd6c9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.577528 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.577499 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:06.767634 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:06.767561 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q"] Jul 23 15:23:06.770175 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:23:06.770136 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b22cc29_2cb5_442c_a14f_84c0992bd6c9.slice/crio-37b4942012e0ae4acaa2c34b495c2d1c9a6227ae3eedc5970efde014ffb58f67 WatchSource:0}: Error finding container 37b4942012e0ae4acaa2c34b495c2d1c9a6227ae3eedc5970efde014ffb58f67: Status 404 returned error can't find the container with id 37b4942012e0ae4acaa2c34b495c2d1c9a6227ae3eedc5970efde014ffb58f67 Jul 23 15:23:07.116700 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:07.116665 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" event={"ID":"0b22cc29-2cb5-442c-a14f-84c0992bd6c9","Type":"ContainerStarted","Data":"37b4942012e0ae4acaa2c34b495c2d1c9a6227ae3eedc5970efde014ffb58f67"} Jul 23 15:23:09.267541 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:09.267509 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:23:09.267790 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:09.267584 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:23:09.267790 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:09.267613 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:23:09.689615 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:09.689573 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w"] Jul 23 15:23:09.692151 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:09.692126 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w" Jul 23 15:23:09.698208 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:09.698172 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 23 15:23:09.698384 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:09.698175 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mwhxd\"" Jul 23 15:23:09.713234 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:09.713209 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w"] Jul 23 15:23:09.786007 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:09.785971 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhjk5\" (UniqueName: \"kubernetes.io/projected/9ede6ef5-5d6c-4523-a2c3-ab77e9d33277-kube-api-access-vhjk5\") pod \"kserve-controller-manager-d6f4b65b8-f2l7w\" (UID: \"9ede6ef5-5d6c-4523-a2c3-ab77e9d33277\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w" Jul 23 15:23:09.786007 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:09.786011 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ede6ef5-5d6c-4523-a2c3-ab77e9d33277-cert\") pod \"kserve-controller-manager-d6f4b65b8-f2l7w\" (UID: \"9ede6ef5-5d6c-4523-a2c3-ab77e9d33277\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w" Jul 23 15:23:09.887317 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:09.887257 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhjk5\" (UniqueName: \"kubernetes.io/projected/9ede6ef5-5d6c-4523-a2c3-ab77e9d33277-kube-api-access-vhjk5\") pod \"kserve-controller-manager-d6f4b65b8-f2l7w\" (UID: \"9ede6ef5-5d6c-4523-a2c3-ab77e9d33277\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w" Jul 23 15:23:09.887510 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:09.887329 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ede6ef5-5d6c-4523-a2c3-ab77e9d33277-cert\") pod \"kserve-controller-manager-d6f4b65b8-f2l7w\" (UID: \"9ede6ef5-5d6c-4523-a2c3-ab77e9d33277\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w" Jul 23 15:23:09.887510 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:23:09.887482 2570 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 23 15:23:09.887616 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:23:09.887559 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ede6ef5-5d6c-4523-a2c3-ab77e9d33277-cert podName:9ede6ef5-5d6c-4523-a2c3-ab77e9d33277 nodeName:}" failed. No retries permitted until 2026-07-23 15:23:10.387536554 +0000 UTC m=+644.003583796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9ede6ef5-5d6c-4523-a2c3-ab77e9d33277-cert") pod "kserve-controller-manager-d6f4b65b8-f2l7w" (UID: "9ede6ef5-5d6c-4523-a2c3-ab77e9d33277") : secret "kserve-webhook-server-cert" not found Jul 23 15:23:09.911323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:09.911255 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhjk5\" (UniqueName: \"kubernetes.io/projected/9ede6ef5-5d6c-4523-a2c3-ab77e9d33277-kube-api-access-vhjk5\") pod \"kserve-controller-manager-d6f4b65b8-f2l7w\" (UID: \"9ede6ef5-5d6c-4523-a2c3-ab77e9d33277\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w" Jul 23 15:23:10.140269 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:10.140229 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" event={"ID":"0b22cc29-2cb5-442c-a14f-84c0992bd6c9","Type":"ContainerStarted","Data":"964a5b0af77840f490d9c697b56a6a8f1b1d0415634fdf923f458cb426064850"} Jul 23 15:23:10.190868 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:10.190810 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" podStartSLOduration=1.695929836 podStartE2EDuration="4.190795437s" totalImagesPullingTime="2.494865601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:23:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:23:10.180542576 +0000 UTC m=+643.796589841" watchObservedRunningTime="2026-07-23 15:23:10.190795437 +0000 UTC m=+643.806842702" Jul 23 15:23:10.391261 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:10.391176 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ede6ef5-5d6c-4523-a2c3-ab77e9d33277-cert\") pod \"kserve-controller-manager-d6f4b65b8-f2l7w\" (UID: \"9ede6ef5-5d6c-4523-a2c3-ab77e9d33277\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w" Jul 23 15:23:10.393564 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:10.393536 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ede6ef5-5d6c-4523-a2c3-ab77e9d33277-cert\") pod \"kserve-controller-manager-d6f4b65b8-f2l7w\" (UID: \"9ede6ef5-5d6c-4523-a2c3-ab77e9d33277\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w" Jul 23 15:23:10.578795 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:10.578682 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:10.597346 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:10.583520 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:10.602663 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:10.602644 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w" Jul 23 15:23:10.734381 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:10.734339 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w"] Jul 23 15:23:10.737008 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:23:10.736971 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ede6ef5_5d6c_4523_a2c3_ab77e9d33277.slice/crio-f80f76498dc77e7b7ab6224699d0abb50aabb1fdff08f1f5b9c5ddd1dd077875 WatchSource:0}: Error finding container f80f76498dc77e7b7ab6224699d0abb50aabb1fdff08f1f5b9c5ddd1dd077875: Status 404 returned error can't find the container with id f80f76498dc77e7b7ab6224699d0abb50aabb1fdff08f1f5b9c5ddd1dd077875 Jul 23 15:23:11.145485 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:11.145452 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w" event={"ID":"9ede6ef5-5d6c-4523-a2c3-ab77e9d33277","Type":"ContainerStarted","Data":"f80f76498dc77e7b7ab6224699d0abb50aabb1fdff08f1f5b9c5ddd1dd077875"} Jul 23 15:23:11.145755 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:11.145741 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:11.146691 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:11.146671 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4p6r6q" Jul 23 15:23:14.160445 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:14.160405 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w" event={"ID":"9ede6ef5-5d6c-4523-a2c3-ab77e9d33277","Type":"ContainerStarted","Data":"9a88dd86c74c5a43c82f8f8a9afcededfadc5ef6cfb16977548bb715042223e6"} Jul 23 15:23:14.160849 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:14.160464 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w" Jul 23 15:23:14.193835 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:14.193784 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w" podStartSLOduration=2.39629387 podStartE2EDuration="5.193769311s" totalImagesPullingTime="2.797475441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:23:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:23:14.192179971 +0000 UTC m=+647.808227235" watchObservedRunningTime="2026-07-23 15:23:14.193769311 +0000 UTC m=+647.809816574" Jul 23 15:23:16.113584 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:16.113553 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-kdk4x" Jul 23 15:23:16.115815 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:16.115794 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-7hg42" Jul 23 15:23:16.816682 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:16.816648 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7"] Jul 23 15:23:16.816847 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:16.816826 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" podUID="e5d575a4-e703-47d7-a632-b65328748130" containerName="discovery" containerID="cri-o://046b3c9b287142a5ff5c5d1e2c4dc95415aaccf9eaed6f2492863063545f8d49" gracePeriod=30 Jul 23 15:23:17.069254 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.069198 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:23:17.149569 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.149534 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2tzx\" (UniqueName: \"kubernetes.io/projected/e5d575a4-e703-47d7-a632-b65328748130-kube-api-access-t2tzx\") pod \"e5d575a4-e703-47d7-a632-b65328748130\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " Jul 23 15:23:17.149955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.149583 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5d575a4-e703-47d7-a632-b65328748130-local-certs\") pod \"e5d575a4-e703-47d7-a632-b65328748130\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " Jul 23 15:23:17.149955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.149601 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5d575a4-e703-47d7-a632-b65328748130-istio-token\") pod \"e5d575a4-e703-47d7-a632-b65328748130\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " Jul 23 15:23:17.149955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.149624 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e5d575a4-e703-47d7-a632-b65328748130-istio-csr-ca-configmap\") pod \"e5d575a4-e703-47d7-a632-b65328748130\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " Jul 23 15:23:17.149955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.149696 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-istio-kubeconfig\") pod \"e5d575a4-e703-47d7-a632-b65328748130\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " Jul 23 15:23:17.149955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.149855 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-istio-csr-dns-cert\") pod \"e5d575a4-e703-47d7-a632-b65328748130\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " Jul 23 15:23:17.149955 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.149901 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-cacerts\") pod \"e5d575a4-e703-47d7-a632-b65328748130\" (UID: \"e5d575a4-e703-47d7-a632-b65328748130\") " Jul 23 15:23:17.150252 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.150083 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5d575a4-e703-47d7-a632-b65328748130-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e5d575a4-e703-47d7-a632-b65328748130" (UID: "e5d575a4-e703-47d7-a632-b65328748130"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:23:17.150252 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.150213 2570 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e5d575a4-e703-47d7-a632-b65328748130-istio-csr-ca-configmap\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:23:17.152122 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.152075 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5d575a4-e703-47d7-a632-b65328748130-local-certs" (OuterVolumeSpecName: "local-certs") pod "e5d575a4-e703-47d7-a632-b65328748130" (UID: "e5d575a4-e703-47d7-a632-b65328748130"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:23:17.152122 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.152077 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5d575a4-e703-47d7-a632-b65328748130-kube-api-access-t2tzx" (OuterVolumeSpecName: "kube-api-access-t2tzx") pod "e5d575a4-e703-47d7-a632-b65328748130" (UID: "e5d575a4-e703-47d7-a632-b65328748130"). InnerVolumeSpecName "kube-api-access-t2tzx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:23:17.152287 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.152167 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e5d575a4-e703-47d7-a632-b65328748130" (UID: "e5d575a4-e703-47d7-a632-b65328748130"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:23:17.152287 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.152214 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5d575a4-e703-47d7-a632-b65328748130-istio-token" (OuterVolumeSpecName: "istio-token") pod "e5d575a4-e703-47d7-a632-b65328748130" (UID: "e5d575a4-e703-47d7-a632-b65328748130"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:23:17.152287 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.152256 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-cacerts" (OuterVolumeSpecName: "cacerts") pod "e5d575a4-e703-47d7-a632-b65328748130" (UID: "e5d575a4-e703-47d7-a632-b65328748130"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:23:17.152431 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.152339 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e5d575a4-e703-47d7-a632-b65328748130" (UID: "e5d575a4-e703-47d7-a632-b65328748130"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:23:17.175907 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.175880 2570 generic.go:356] "Generic (PLEG): container finished" podID="e5d575a4-e703-47d7-a632-b65328748130" containerID="046b3c9b287142a5ff5c5d1e2c4dc95415aaccf9eaed6f2492863063545f8d49" exitCode=0 Jul 23 15:23:17.176020 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.175930 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" Jul 23 15:23:17.176020 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.175932 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" event={"ID":"e5d575a4-e703-47d7-a632-b65328748130","Type":"ContainerDied","Data":"046b3c9b287142a5ff5c5d1e2c4dc95415aaccf9eaed6f2492863063545f8d49"} Jul 23 15:23:17.176093 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.176034 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7" event={"ID":"e5d575a4-e703-47d7-a632-b65328748130","Type":"ContainerDied","Data":"98279adadf0420575e87ebe9987ba04d2fd0c3e054b19b4ef5e3e9ad439b958c"} Jul 23 15:23:17.176093 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.176057 2570 scope.go:117] "RemoveContainer" containerID="046b3c9b287142a5ff5c5d1e2c4dc95415aaccf9eaed6f2492863063545f8d49" Jul 23 15:23:17.185523 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.185507 2570 scope.go:117] "RemoveContainer" containerID="046b3c9b287142a5ff5c5d1e2c4dc95415aaccf9eaed6f2492863063545f8d49" Jul 23 15:23:17.185755 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:23:17.185727 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"046b3c9b287142a5ff5c5d1e2c4dc95415aaccf9eaed6f2492863063545f8d49\": container with ID starting with 046b3c9b287142a5ff5c5d1e2c4dc95415aaccf9eaed6f2492863063545f8d49 not found: ID does not exist" containerID="046b3c9b287142a5ff5c5d1e2c4dc95415aaccf9eaed6f2492863063545f8d49" Jul 23 15:23:17.185844 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.185765 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"046b3c9b287142a5ff5c5d1e2c4dc95415aaccf9eaed6f2492863063545f8d49"} err="failed to get container status \"046b3c9b287142a5ff5c5d1e2c4dc95415aaccf9eaed6f2492863063545f8d49\": rpc error: code = NotFound desc = could not find container \"046b3c9b287142a5ff5c5d1e2c4dc95415aaccf9eaed6f2492863063545f8d49\": container with ID starting with 046b3c9b287142a5ff5c5d1e2c4dc95415aaccf9eaed6f2492863063545f8d49 not found: ID does not exist" Jul 23 15:23:17.251081 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.251050 2570 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-cacerts\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:23:17.251081 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.251075 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2tzx\" (UniqueName: \"kubernetes.io/projected/e5d575a4-e703-47d7-a632-b65328748130-kube-api-access-t2tzx\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:23:17.251081 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.251086 2570 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5d575a4-e703-47d7-a632-b65328748130-local-certs\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:23:17.251369 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.251097 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5d575a4-e703-47d7-a632-b65328748130-istio-token\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:23:17.251369 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.251105 2570 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-istio-kubeconfig\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:23:17.251369 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.251112 2570 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e5d575a4-e703-47d7-a632-b65328748130-istio-csr-dns-cert\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:23:17.251369 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.251163 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7"] Jul 23 15:23:17.258893 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:17.258866 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-jnbp7"] Jul 23 15:23:18.438766 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:18.438736 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5d575a4-e703-47d7-a632-b65328748130" path="/var/lib/kubelet/pods/e5d575a4-e703-47d7-a632-b65328748130/volumes" Jul 23 15:23:45.169972 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:45.169942 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2l7w" Jul 23 15:23:58.795572 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:58.795537 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h7vfp"] Jul 23 15:23:58.795962 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:58.795871 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5d575a4-e703-47d7-a632-b65328748130" containerName="discovery" Jul 23 15:23:58.795962 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:58.795883 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5d575a4-e703-47d7-a632-b65328748130" containerName="discovery" Jul 23 15:23:58.795962 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:58.795943 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5d575a4-e703-47d7-a632-b65328748130" containerName="discovery" Jul 23 15:23:58.798222 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:58.798201 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h7vfp" Jul 23 15:23:58.800995 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:58.800972 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 15:23:58.801131 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:58.801050 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-djtw4\"" Jul 23 15:23:58.801131 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:58.801073 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 15:23:58.808754 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:58.808734 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h7vfp"] Jul 23 15:23:58.911546 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:58.911519 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk7g4\" (UniqueName: \"kubernetes.io/projected/5b026a23-30b8-440f-9172-f868c42d18f1-kube-api-access-pk7g4\") pod \"kuadrant-operator-catalog-h7vfp\" (UID: \"5b026a23-30b8-440f-9172-f868c42d18f1\") " pod="kuadrant-system/kuadrant-operator-catalog-h7vfp" Jul 23 15:23:59.012743 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:59.012709 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk7g4\" (UniqueName: \"kubernetes.io/projected/5b026a23-30b8-440f-9172-f868c42d18f1-kube-api-access-pk7g4\") pod \"kuadrant-operator-catalog-h7vfp\" (UID: \"5b026a23-30b8-440f-9172-f868c42d18f1\") " pod="kuadrant-system/kuadrant-operator-catalog-h7vfp" Jul 23 15:23:59.022013 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:59.021985 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk7g4\" (UniqueName: \"kubernetes.io/projected/5b026a23-30b8-440f-9172-f868c42d18f1-kube-api-access-pk7g4\") pod \"kuadrant-operator-catalog-h7vfp\" (UID: \"5b026a23-30b8-440f-9172-f868c42d18f1\") " pod="kuadrant-system/kuadrant-operator-catalog-h7vfp" Jul 23 15:23:59.110394 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:59.110351 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h7vfp" Jul 23 15:23:59.152868 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:59.152832 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h7vfp"] Jul 23 15:23:59.240182 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:59.240149 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h7vfp"] Jul 23 15:23:59.243033 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:23:59.243003 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b026a23_30b8_440f_9172_f868c42d18f1.slice/crio-29647ba29c9f3a5fcddb8eb5de4f88db1fd828feeeadf70a2c94b5183917331b WatchSource:0}: Error finding container 29647ba29c9f3a5fcddb8eb5de4f88db1fd828feeeadf70a2c94b5183917331b: Status 404 returned error can't find the container with id 29647ba29c9f3a5fcddb8eb5de4f88db1fd828feeeadf70a2c94b5183917331b Jul 23 15:23:59.357481 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:59.357444 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h7vfp" event={"ID":"5b026a23-30b8-440f-9172-f868c42d18f1","Type":"ContainerStarted","Data":"29647ba29c9f3a5fcddb8eb5de4f88db1fd828feeeadf70a2c94b5183917331b"} Jul 23 15:23:59.364604 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:59.364545 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-djtvm"] Jul 23 15:23:59.367420 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:59.367406 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-djtvm" Jul 23 15:23:59.377160 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:59.377142 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-djtvm"] Jul 23 15:23:59.517712 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:59.517681 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d876m\" (UniqueName: \"kubernetes.io/projected/aa23ceb8-ae48-4455-be26-5fa54d650ebe-kube-api-access-d876m\") pod \"kuadrant-operator-catalog-djtvm\" (UID: \"aa23ceb8-ae48-4455-be26-5fa54d650ebe\") " pod="kuadrant-system/kuadrant-operator-catalog-djtvm" Jul 23 15:23:59.618685 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:59.618594 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d876m\" (UniqueName: \"kubernetes.io/projected/aa23ceb8-ae48-4455-be26-5fa54d650ebe-kube-api-access-d876m\") pod \"kuadrant-operator-catalog-djtvm\" (UID: \"aa23ceb8-ae48-4455-be26-5fa54d650ebe\") " pod="kuadrant-system/kuadrant-operator-catalog-djtvm" Jul 23 15:23:59.628952 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:59.628924 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d876m\" (UniqueName: \"kubernetes.io/projected/aa23ceb8-ae48-4455-be26-5fa54d650ebe-kube-api-access-d876m\") pod \"kuadrant-operator-catalog-djtvm\" (UID: \"aa23ceb8-ae48-4455-be26-5fa54d650ebe\") " pod="kuadrant-system/kuadrant-operator-catalog-djtvm" Jul 23 15:23:59.676924 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:59.676887 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-djtvm" Jul 23 15:23:59.812650 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:23:59.812625 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-djtvm"] Jul 23 15:23:59.814685 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:23:59.814661 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa23ceb8_ae48_4455_be26_5fa54d650ebe.slice/crio-73c49306b3a90af33ae285b092bc556955a617ec75ec1b4be98f5ca8a11adb6b WatchSource:0}: Error finding container 73c49306b3a90af33ae285b092bc556955a617ec75ec1b4be98f5ca8a11adb6b: Status 404 returned error can't find the container with id 73c49306b3a90af33ae285b092bc556955a617ec75ec1b4be98f5ca8a11adb6b Jul 23 15:24:00.364055 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:00.364023 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-djtvm" event={"ID":"aa23ceb8-ae48-4455-be26-5fa54d650ebe","Type":"ContainerStarted","Data":"73c49306b3a90af33ae285b092bc556955a617ec75ec1b4be98f5ca8a11adb6b"} Jul 23 15:24:02.373393 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:02.373355 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h7vfp" event={"ID":"5b026a23-30b8-440f-9172-f868c42d18f1","Type":"ContainerStarted","Data":"7685d43971408fe36c6ada637f27c3611c5a529819feba84feb4ed9f87da8086"} Jul 23 15:24:02.373838 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:02.373387 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-h7vfp" podUID="5b026a23-30b8-440f-9172-f868c42d18f1" containerName="registry-server" containerID="cri-o://7685d43971408fe36c6ada637f27c3611c5a529819feba84feb4ed9f87da8086" gracePeriod=2 Jul 23 15:24:02.374805 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:02.374783 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-djtvm" event={"ID":"aa23ceb8-ae48-4455-be26-5fa54d650ebe","Type":"ContainerStarted","Data":"3a0884912d5fc463836cf9875436210e04fbcd7ff2acc9b23521dc902466a554"} Jul 23 15:24:02.395841 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:02.395794 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-h7vfp" podStartSLOduration=2.307304573 podStartE2EDuration="4.395778308s" totalImagesPullingTime="2.088473735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:24:02.393477248 +0000 UTC m=+696.009524513" watchObservedRunningTime="2026-07-23 15:24:02.395778308 +0000 UTC m=+696.011825571" Jul 23 15:24:02.412984 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:02.412929 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-djtvm" podStartSLOduration=1.893415873 podStartE2EDuration="3.412911683s" totalImagesPullingTime="1.51949581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:23:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:24:02.411825966 +0000 UTC m=+696.027873231" watchObservedRunningTime="2026-07-23 15:24:02.412911683 +0000 UTC m=+696.028958946" Jul 23 15:24:02.619470 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:02.619440 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h7vfp" Jul 23 15:24:02.647070 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:02.646993 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk7g4\" (UniqueName: \"kubernetes.io/projected/5b026a23-30b8-440f-9172-f868c42d18f1-kube-api-access-pk7g4\") pod \"5b026a23-30b8-440f-9172-f868c42d18f1\" (UID: \"5b026a23-30b8-440f-9172-f868c42d18f1\") " Jul 23 15:24:02.649102 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:02.649076 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b026a23-30b8-440f-9172-f868c42d18f1-kube-api-access-pk7g4" (OuterVolumeSpecName: "kube-api-access-pk7g4") pod "5b026a23-30b8-440f-9172-f868c42d18f1" (UID: "5b026a23-30b8-440f-9172-f868c42d18f1"). InnerVolumeSpecName "kube-api-access-pk7g4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:24:02.748084 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:02.748044 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk7g4\" (UniqueName: \"kubernetes.io/projected/5b026a23-30b8-440f-9172-f868c42d18f1-kube-api-access-pk7g4\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:24:03.380069 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:03.380035 2570 generic.go:356] "Generic (PLEG): container finished" podID="5b026a23-30b8-440f-9172-f868c42d18f1" containerID="7685d43971408fe36c6ada637f27c3611c5a529819feba84feb4ed9f87da8086" exitCode=0 Jul 23 15:24:03.380479 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:03.380077 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h7vfp" Jul 23 15:24:03.380479 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:03.380118 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h7vfp" event={"ID":"5b026a23-30b8-440f-9172-f868c42d18f1","Type":"ContainerDied","Data":"7685d43971408fe36c6ada637f27c3611c5a529819feba84feb4ed9f87da8086"} Jul 23 15:24:03.380479 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:03.380168 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h7vfp" event={"ID":"5b026a23-30b8-440f-9172-f868c42d18f1","Type":"ContainerDied","Data":"29647ba29c9f3a5fcddb8eb5de4f88db1fd828feeeadf70a2c94b5183917331b"} Jul 23 15:24:03.380479 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:03.380188 2570 scope.go:117] "RemoveContainer" containerID="7685d43971408fe36c6ada637f27c3611c5a529819feba84feb4ed9f87da8086" Jul 23 15:24:03.388819 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:03.388801 2570 scope.go:117] "RemoveContainer" containerID="7685d43971408fe36c6ada637f27c3611c5a529819feba84feb4ed9f87da8086" Jul 23 15:24:03.389087 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:24:03.389071 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7685d43971408fe36c6ada637f27c3611c5a529819feba84feb4ed9f87da8086\": container with ID starting with 7685d43971408fe36c6ada637f27c3611c5a529819feba84feb4ed9f87da8086 not found: ID does not exist" containerID="7685d43971408fe36c6ada637f27c3611c5a529819feba84feb4ed9f87da8086" Jul 23 15:24:03.389156 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:03.389100 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7685d43971408fe36c6ada637f27c3611c5a529819feba84feb4ed9f87da8086"} err="failed to get container status \"7685d43971408fe36c6ada637f27c3611c5a529819feba84feb4ed9f87da8086\": rpc error: code = NotFound desc = could not find container \"7685d43971408fe36c6ada637f27c3611c5a529819feba84feb4ed9f87da8086\": container with ID starting with 7685d43971408fe36c6ada637f27c3611c5a529819feba84feb4ed9f87da8086 not found: ID does not exist" Jul 23 15:24:03.402585 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:03.402553 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h7vfp"] Jul 23 15:24:03.408843 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:03.408818 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h7vfp"] Jul 23 15:24:04.436985 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:04.436955 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b026a23-30b8-440f-9172-f868c42d18f1" path="/var/lib/kubelet/pods/5b026a23-30b8-440f-9172-f868c42d18f1/volumes" Jul 23 15:24:09.677827 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:09.677795 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-djtvm" Jul 23 15:24:09.678257 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:09.677852 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-djtvm" Jul 23 15:24:09.700769 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:09.700741 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-djtvm" Jul 23 15:24:10.437933 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:10.437909 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-djtvm" Jul 23 15:24:31.085887 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:31.085855 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8w6hg"] Jul 23 15:24:31.086285 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:31.086199 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b026a23-30b8-440f-9172-f868c42d18f1" containerName="registry-server" Jul 23 15:24:31.086285 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:31.086211 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b026a23-30b8-440f-9172-f868c42d18f1" containerName="registry-server" Jul 23 15:24:31.086285 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:31.086265 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b026a23-30b8-440f-9172-f868c42d18f1" containerName="registry-server" Jul 23 15:24:31.090378 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:31.090357 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8w6hg" Jul 23 15:24:31.093031 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:31.093011 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-hrn54\"" Jul 23 15:24:31.102648 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:31.102627 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8w6hg"] Jul 23 15:24:31.196467 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:31.196436 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2njl9\" (UniqueName: \"kubernetes.io/projected/8828e7d3-2224-4a34-bbd9-1f5faf9dbdfa-kube-api-access-2njl9\") pod \"authorino-operator-75cc8d5cb7-8w6hg\" (UID: \"8828e7d3-2224-4a34-bbd9-1f5faf9dbdfa\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8w6hg" Jul 23 15:24:31.297689 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:31.297651 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2njl9\" (UniqueName: \"kubernetes.io/projected/8828e7d3-2224-4a34-bbd9-1f5faf9dbdfa-kube-api-access-2njl9\") pod \"authorino-operator-75cc8d5cb7-8w6hg\" (UID: \"8828e7d3-2224-4a34-bbd9-1f5faf9dbdfa\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8w6hg" Jul 23 15:24:31.310868 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:31.310841 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2njl9\" (UniqueName: \"kubernetes.io/projected/8828e7d3-2224-4a34-bbd9-1f5faf9dbdfa-kube-api-access-2njl9\") pod \"authorino-operator-75cc8d5cb7-8w6hg\" (UID: \"8828e7d3-2224-4a34-bbd9-1f5faf9dbdfa\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8w6hg" Jul 23 15:24:31.400106 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:31.400066 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8w6hg" Jul 23 15:24:31.535562 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:24:31.535529 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8828e7d3_2224_4a34_bbd9_1f5faf9dbdfa.slice/crio-ebee319efc1d6936b977a6f8395cc49c26c4a7a01c9a6a1a53b5502b87abf28f WatchSource:0}: Error finding container ebee319efc1d6936b977a6f8395cc49c26c4a7a01c9a6a1a53b5502b87abf28f: Status 404 returned error can't find the container with id ebee319efc1d6936b977a6f8395cc49c26c4a7a01c9a6a1a53b5502b87abf28f Jul 23 15:24:31.537182 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:31.537156 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8w6hg"] Jul 23 15:24:32.506621 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:32.506577 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8w6hg" event={"ID":"8828e7d3-2224-4a34-bbd9-1f5faf9dbdfa","Type":"ContainerStarted","Data":"ebee319efc1d6936b977a6f8395cc49c26c4a7a01c9a6a1a53b5502b87abf28f"} Jul 23 15:24:34.517507 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:34.517477 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8w6hg" event={"ID":"8828e7d3-2224-4a34-bbd9-1f5faf9dbdfa","Type":"ContainerStarted","Data":"9decb6bc5834c737063a8a63c768089dbdec21a6343bd738659db7b3dce90578"} Jul 23 15:24:34.517884 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:34.517607 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8w6hg" Jul 23 15:24:34.615532 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:34.615484 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8w6hg" podStartSLOduration=1.378336848 podStartE2EDuration="3.615462327s" totalImagesPullingTime="2.237125479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:24:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:24:34.613360544 +0000 UTC m=+728.229407808" watchObservedRunningTime="2026-07-23 15:24:34.615462327 +0000 UTC m=+728.231509591" Jul 23 15:24:34.683586 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:34.683551 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-psr4q"] Jul 23 15:24:34.686099 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:34.686081 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psr4q" Jul 23 15:24:34.689682 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:34.689658 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-59sdx\"" Jul 23 15:24:34.692346 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:34.692324 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 23 15:24:34.708370 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:34.708348 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-psr4q"] Jul 23 15:24:34.826511 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:34.826422 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hzn7\" (UniqueName: \"kubernetes.io/projected/42a32983-bbde-41bc-a3f1-efb28b298737-kube-api-access-8hzn7\") pod \"dns-operator-controller-manager-864dd78c4c-psr4q\" (UID: \"42a32983-bbde-41bc-a3f1-efb28b298737\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psr4q" Jul 23 15:24:34.927547 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:34.927512 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hzn7\" (UniqueName: \"kubernetes.io/projected/42a32983-bbde-41bc-a3f1-efb28b298737-kube-api-access-8hzn7\") pod \"dns-operator-controller-manager-864dd78c4c-psr4q\" (UID: \"42a32983-bbde-41bc-a3f1-efb28b298737\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psr4q" Jul 23 15:24:34.939961 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:34.939932 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hzn7\" (UniqueName: \"kubernetes.io/projected/42a32983-bbde-41bc-a3f1-efb28b298737-kube-api-access-8hzn7\") pod \"dns-operator-controller-manager-864dd78c4c-psr4q\" (UID: \"42a32983-bbde-41bc-a3f1-efb28b298737\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psr4q" Jul 23 15:24:34.996031 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:34.995998 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psr4q" Jul 23 15:24:35.155617 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:35.155581 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-psr4q"] Jul 23 15:24:35.155741 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:24:35.155582 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a32983_bbde_41bc_a3f1_efb28b298737.slice/crio-997471ddeaa027ef40651391bd5f1a49613ddd93f6a8f666f3bf8a456fcb550d WatchSource:0}: Error finding container 997471ddeaa027ef40651391bd5f1a49613ddd93f6a8f666f3bf8a456fcb550d: Status 404 returned error can't find the container with id 997471ddeaa027ef40651391bd5f1a49613ddd93f6a8f666f3bf8a456fcb550d Jul 23 15:24:35.523991 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:35.523956 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psr4q" event={"ID":"42a32983-bbde-41bc-a3f1-efb28b298737","Type":"ContainerStarted","Data":"997471ddeaa027ef40651391bd5f1a49613ddd93f6a8f666f3bf8a456fcb550d"} Jul 23 15:24:37.533948 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:37.533918 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psr4q" event={"ID":"42a32983-bbde-41bc-a3f1-efb28b298737","Type":"ContainerStarted","Data":"dedbe1decb2e36867a4da84d390887372fc162bae3af034ae730a1ff3327d193"} Jul 23 15:24:37.534239 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:37.534011 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psr4q" Jul 23 15:24:37.558019 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:37.557960 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psr4q" podStartSLOduration=1.264614731 podStartE2EDuration="3.557943902s" totalImagesPullingTime="2.293329171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:24:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:24:37.557319201 +0000 UTC m=+731.173366469" watchObservedRunningTime="2026-07-23 15:24:37.557943902 +0000 UTC m=+731.173991165" Jul 23 15:24:39.550842 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:39.550805 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks"] Jul 23 15:24:39.554209 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:39.554192 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:24:39.558358 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:39.558337 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fvhv4\"" Jul 23 15:24:39.574549 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:39.574523 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks"] Jul 23 15:24:39.672913 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:39.672881 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt8kq\" (UniqueName: \"kubernetes.io/projected/51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f-kube-api-access-lt8kq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fq6ks\" (UID: \"51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:24:39.673083 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:39.672924 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fq6ks\" (UID: \"51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:24:39.774248 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:39.774215 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt8kq\" (UniqueName: \"kubernetes.io/projected/51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f-kube-api-access-lt8kq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fq6ks\" (UID: \"51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:24:39.774431 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:39.774266 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fq6ks\" (UID: \"51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:24:39.774708 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:39.774687 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fq6ks\" (UID: \"51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:24:39.798674 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:39.798650 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt8kq\" (UniqueName: \"kubernetes.io/projected/51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f-kube-api-access-lt8kq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fq6ks\" (UID: \"51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:24:39.864123 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:39.864089 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:24:40.020618 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:40.020583 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks"] Jul 23 15:24:40.023800 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:24:40.023770 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51ca8b0a_1163_4e13_8e5c_5b6a502a8e0f.slice/crio-26db65db3a0724c7e5ec2c3cb410488107e3b22807498b0a932616f1f8cc02e6 WatchSource:0}: Error finding container 26db65db3a0724c7e5ec2c3cb410488107e3b22807498b0a932616f1f8cc02e6: Status 404 returned error can't find the container with id 26db65db3a0724c7e5ec2c3cb410488107e3b22807498b0a932616f1f8cc02e6 Jul 23 15:24:40.547590 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:40.547558 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" event={"ID":"51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f","Type":"ContainerStarted","Data":"26db65db3a0724c7e5ec2c3cb410488107e3b22807498b0a932616f1f8cc02e6"} Jul 23 15:24:41.554528 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:41.554499 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh"] Jul 23 15:24:41.557975 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:41.557955 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:24:41.561154 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:41.561132 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-pkr42\"" Jul 23 15:24:41.578792 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:41.578770 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh"] Jul 23 15:24:41.693743 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:41.693715 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdb4j\" (UniqueName: \"kubernetes.io/projected/98445cd4-1ff3-4205-86c0-a608453704dc-kube-api-access-kdb4j\") pod \"limitador-operator-controller-manager-859d7b78bd-c6cjh\" (UID: \"98445cd4-1ff3-4205-86c0-a608453704dc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:24:41.794195 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:41.794156 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdb4j\" (UniqueName: \"kubernetes.io/projected/98445cd4-1ff3-4205-86c0-a608453704dc-kube-api-access-kdb4j\") pod \"limitador-operator-controller-manager-859d7b78bd-c6cjh\" (UID: \"98445cd4-1ff3-4205-86c0-a608453704dc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:24:41.810494 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:41.810374 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdb4j\" (UniqueName: \"kubernetes.io/projected/98445cd4-1ff3-4205-86c0-a608453704dc-kube-api-access-kdb4j\") pod \"limitador-operator-controller-manager-859d7b78bd-c6cjh\" (UID: \"98445cd4-1ff3-4205-86c0-a608453704dc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:24:41.867872 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:41.867836 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:24:42.010515 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:42.010490 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh"] Jul 23 15:24:42.013159 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:24:42.013130 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98445cd4_1ff3_4205_86c0_a608453704dc.slice/crio-64abd770e8d8ada8b00f770846ee291076107c7d762171b4171c757bb0e7635f WatchSource:0}: Error finding container 64abd770e8d8ada8b00f770846ee291076107c7d762171b4171c757bb0e7635f: Status 404 returned error can't find the container with id 64abd770e8d8ada8b00f770846ee291076107c7d762171b4171c757bb0e7635f Jul 23 15:24:42.558336 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:42.558283 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" event={"ID":"98445cd4-1ff3-4205-86c0-a608453704dc","Type":"ContainerStarted","Data":"64abd770e8d8ada8b00f770846ee291076107c7d762171b4171c757bb0e7635f"} Jul 23 15:24:44.570370 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:44.570324 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" event={"ID":"98445cd4-1ff3-4205-86c0-a608453704dc","Type":"ContainerStarted","Data":"b0de9d31787a2a489b4d542ba7f2e33faa88c10b81830fc27da4dc8de8dd13bd"} Jul 23 15:24:44.570776 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:44.570444 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:24:44.591420 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:44.591357 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" podStartSLOduration=1.8337029679999999 podStartE2EDuration="3.591341917s" totalImagesPullingTime="1.757638949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:24:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:24:44.590615662 +0000 UTC m=+738.206662928" watchObservedRunningTime="2026-07-23 15:24:44.591341917 +0000 UTC m=+738.207389181" Jul 23 15:24:45.526535 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:45.526504 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8w6hg" Jul 23 15:24:46.588035 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:46.588003 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" event={"ID":"51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f","Type":"ContainerStarted","Data":"98eac5b678c9a7a91da0fb6e1a6d6e9e41016d48d830ee13880ddd06c8de5b70"} Jul 23 15:24:46.588487 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:46.588126 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:24:48.540823 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.540794 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-psr4q" Jul 23 15:24:48.541217 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.541044 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" podStartSLOduration=4.029618582 podStartE2EDuration="9.541028106s" totalImagesPullingTime="5.511409524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:24:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:24:46.617207539 +0000 UTC m=+740.233254803" watchObservedRunningTime="2026-07-23 15:24:48.541028106 +0000 UTC m=+742.157075371" Jul 23 15:24:48.541571 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.541556 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4"] Jul 23 15:24:48.544992 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.544977 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" Jul 23 15:24:48.551773 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.551753 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 23 15:24:48.551871 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.551839 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 23 15:24:48.551932 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.551882 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4h6zx\"" Jul 23 15:24:48.559139 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.559119 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/78cbfd02-7da4-4ecd-8416-fe34927d9c63-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zh7f4\" (UID: \"78cbfd02-7da4-4ecd-8416-fe34927d9c63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" Jul 23 15:24:48.559251 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.559232 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk8bx\" (UniqueName: \"kubernetes.io/projected/78cbfd02-7da4-4ecd-8416-fe34927d9c63-kube-api-access-pk8bx\") pod \"kuadrant-console-plugin-66b5774985-zh7f4\" (UID: \"78cbfd02-7da4-4ecd-8416-fe34927d9c63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" Jul 23 15:24:48.559331 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.559288 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/78cbfd02-7da4-4ecd-8416-fe34927d9c63-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zh7f4\" (UID: \"78cbfd02-7da4-4ecd-8416-fe34927d9c63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" Jul 23 15:24:48.564231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.564207 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4"] Jul 23 15:24:48.659857 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.659820 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk8bx\" (UniqueName: \"kubernetes.io/projected/78cbfd02-7da4-4ecd-8416-fe34927d9c63-kube-api-access-pk8bx\") pod \"kuadrant-console-plugin-66b5774985-zh7f4\" (UID: \"78cbfd02-7da4-4ecd-8416-fe34927d9c63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" Jul 23 15:24:48.660000 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.659866 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/78cbfd02-7da4-4ecd-8416-fe34927d9c63-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zh7f4\" (UID: \"78cbfd02-7da4-4ecd-8416-fe34927d9c63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" Jul 23 15:24:48.660000 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.659907 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/78cbfd02-7da4-4ecd-8416-fe34927d9c63-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zh7f4\" (UID: \"78cbfd02-7da4-4ecd-8416-fe34927d9c63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" Jul 23 15:24:48.660100 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:24:48.660016 2570 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 23 15:24:48.660134 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:24:48.660100 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78cbfd02-7da4-4ecd-8416-fe34927d9c63-plugin-serving-cert podName:78cbfd02-7da4-4ecd-8416-fe34927d9c63 nodeName:}" failed. No retries permitted until 2026-07-23 15:24:49.160083356 +0000 UTC m=+742.776130602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/78cbfd02-7da4-4ecd-8416-fe34927d9c63-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-zh7f4" (UID: "78cbfd02-7da4-4ecd-8416-fe34927d9c63") : secret "plugin-serving-cert" not found Jul 23 15:24:48.660580 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.660563 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/78cbfd02-7da4-4ecd-8416-fe34927d9c63-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zh7f4\" (UID: \"78cbfd02-7da4-4ecd-8416-fe34927d9c63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" Jul 23 15:24:48.672627 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:48.672605 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk8bx\" (UniqueName: \"kubernetes.io/projected/78cbfd02-7da4-4ecd-8416-fe34927d9c63-kube-api-access-pk8bx\") pod \"kuadrant-console-plugin-66b5774985-zh7f4\" (UID: \"78cbfd02-7da4-4ecd-8416-fe34927d9c63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" Jul 23 15:24:49.164171 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:49.164139 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/78cbfd02-7da4-4ecd-8416-fe34927d9c63-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zh7f4\" (UID: \"78cbfd02-7da4-4ecd-8416-fe34927d9c63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" Jul 23 15:24:49.166513 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:49.166492 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/78cbfd02-7da4-4ecd-8416-fe34927d9c63-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zh7f4\" (UID: \"78cbfd02-7da4-4ecd-8416-fe34927d9c63\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" Jul 23 15:24:49.454326 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:49.454203 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" Jul 23 15:24:49.588212 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:49.588179 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4"] Jul 23 15:24:49.591443 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:24:49.591415 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78cbfd02_7da4_4ecd_8416_fe34927d9c63.slice/crio-e923526466ccc41994625fa9e71adc17658417cc36b8048a50f2416eeb680c05 WatchSource:0}: Error finding container e923526466ccc41994625fa9e71adc17658417cc36b8048a50f2416eeb680c05: Status 404 returned error can't find the container with id e923526466ccc41994625fa9e71adc17658417cc36b8048a50f2416eeb680c05 Jul 23 15:24:49.603685 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:49.603658 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" event={"ID":"78cbfd02-7da4-4ecd-8416-fe34927d9c63","Type":"ContainerStarted","Data":"e923526466ccc41994625fa9e71adc17658417cc36b8048a50f2416eeb680c05"} Jul 23 15:24:55.585002 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:55.584970 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:24:57.595215 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:57.595181 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:24:59.354475 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.354439 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks"] Jul 23 15:24:59.355095 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.354652 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" podUID="51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" containerName="manager" containerID="cri-o://98eac5b678c9a7a91da0fb6e1a6d6e9e41016d48d830ee13880ddd06c8de5b70" gracePeriod=2 Jul 23 15:24:59.366191 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.366161 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks"] Jul 23 15:24:59.383665 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.383557 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n"] Jul 23 15:24:59.384121 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.384105 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" containerName="manager" Jul 23 15:24:59.384209 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.384125 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" containerName="manager" Jul 23 15:24:59.384264 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.384230 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" containerName="manager" Jul 23 15:24:59.387279 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.387210 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh"] Jul 23 15:24:59.387423 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.387329 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" Jul 23 15:24:59.387491 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.387417 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" podUID="98445cd4-1ff3-4205-86c0-a608453704dc" containerName="manager" containerID="cri-o://b0de9d31787a2a489b4d542ba7f2e33faa88c10b81830fc27da4dc8de8dd13bd" gracePeriod=2 Jul 23 15:24:59.389926 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:24:59.389892 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fq6ks\" is forbidden: User \"system:node:ip-10-0-129-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-63.ec2.internal' and this object" podUID="51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:24:59.405288 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.405207 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n"] Jul 23 15:24:59.407523 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.407489 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh"] Jul 23 15:24:59.413773 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.413746 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqc5l"] Jul 23 15:24:59.414266 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.414246 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98445cd4-1ff3-4205-86c0-a608453704dc" containerName="manager" Jul 23 15:24:59.414266 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.414265 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="98445cd4-1ff3-4205-86c0-a608453704dc" containerName="manager" Jul 23 15:24:59.414428 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.414387 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="98445cd4-1ff3-4205-86c0-a608453704dc" containerName="manager" Jul 23 15:24:59.423641 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:24:59.419970 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-c6cjh\" is forbidden: User \"system:node:ip-10-0-129-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-63.ec2.internal' and this object" podUID="98445cd4-1ff3-4205-86c0-a608453704dc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:24:59.423641 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.421626 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqc5l" Jul 23 15:24:59.423641 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:24:59.421895 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fq6ks\" is forbidden: User \"system:node:ip-10-0-129-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-63.ec2.internal' and this object" podUID="51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:24:59.423904 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:24:59.423838 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-c6cjh\" is forbidden: User \"system:node:ip-10-0-129-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-63.ec2.internal' and this object" podUID="98445cd4-1ff3-4205-86c0-a608453704dc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:24:59.430241 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.429607 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqc5l"] Jul 23 15:24:59.451594 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:24:59.451549 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-c6cjh\" is forbidden: User \"system:node:ip-10-0-129-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-63.ec2.internal' and this object" podUID="98445cd4-1ff3-4205-86c0-a608453704dc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:24:59.453700 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:24:59.453643 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fq6ks\" is forbidden: User \"system:node:ip-10-0-129-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-63.ec2.internal' and this object" podUID="51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:24:59.462324 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.462286 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvd7m\" (UniqueName: \"kubernetes.io/projected/9f81edca-0ad4-458f-93b0-810787b6e2f5-kube-api-access-gvd7m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-txt8n\" (UID: \"9f81edca-0ad4-458f-93b0-810787b6e2f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" Jul 23 15:24:59.462448 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.462396 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f81edca-0ad4-458f-93b0-810787b6e2f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-txt8n\" (UID: \"9f81edca-0ad4-458f-93b0-810787b6e2f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" Jul 23 15:24:59.563139 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.563104 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvd7m\" (UniqueName: \"kubernetes.io/projected/9f81edca-0ad4-458f-93b0-810787b6e2f5-kube-api-access-gvd7m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-txt8n\" (UID: \"9f81edca-0ad4-458f-93b0-810787b6e2f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" Jul 23 15:24:59.563343 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.563150 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nprb\" (UniqueName: \"kubernetes.io/projected/263bc01f-e1e1-4329-9c95-2785b4818ede-kube-api-access-8nprb\") pod \"limitador-operator-controller-manager-859d7b78bd-bqc5l\" (UID: \"263bc01f-e1e1-4329-9c95-2785b4818ede\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqc5l" Jul 23 15:24:59.563343 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.563188 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f81edca-0ad4-458f-93b0-810787b6e2f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-txt8n\" (UID: \"9f81edca-0ad4-458f-93b0-810787b6e2f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" Jul 23 15:24:59.563595 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.563561 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f81edca-0ad4-458f-93b0-810787b6e2f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-txt8n\" (UID: \"9f81edca-0ad4-458f-93b0-810787b6e2f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" Jul 23 15:24:59.578967 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.578937 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvd7m\" (UniqueName: \"kubernetes.io/projected/9f81edca-0ad4-458f-93b0-810787b6e2f5-kube-api-access-gvd7m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-txt8n\" (UID: \"9f81edca-0ad4-458f-93b0-810787b6e2f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" Jul 23 15:24:59.664338 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.664244 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nprb\" (UniqueName: \"kubernetes.io/projected/263bc01f-e1e1-4329-9c95-2785b4818ede-kube-api-access-8nprb\") pod \"limitador-operator-controller-manager-859d7b78bd-bqc5l\" (UID: \"263bc01f-e1e1-4329-9c95-2785b4818ede\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqc5l" Jul 23 15:24:59.676558 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.676526 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nprb\" (UniqueName: \"kubernetes.io/projected/263bc01f-e1e1-4329-9c95-2785b4818ede-kube-api-access-8nprb\") pod \"limitador-operator-controller-manager-859d7b78bd-bqc5l\" (UID: \"263bc01f-e1e1-4329-9c95-2785b4818ede\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqc5l" Jul 23 15:24:59.766095 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.766054 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" Jul 23 15:24:59.774178 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:24:59.774148 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqc5l" Jul 23 15:25:12.307677 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.307645 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n"] Jul 23 15:25:12.324045 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.324019 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:25:12.326601 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:25:12.326573 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fq6ks\" is forbidden: User \"system:node:ip-10-0-129-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-63.ec2.internal' and this object" podUID="51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:25:12.484674 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.484637 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f-extensions-socket-volume\") pod \"51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f\" (UID: \"51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f\") " Jul 23 15:25:12.484847 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.484715 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lt8kq\" (UniqueName: \"kubernetes.io/projected/51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f-kube-api-access-lt8kq\") pod \"51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f\" (UID: \"51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f\") " Jul 23 15:25:12.485190 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.485163 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" (UID: "51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:25:12.486812 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.486791 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f-kube-api-access-lt8kq" (OuterVolumeSpecName: "kube-api-access-lt8kq") pod "51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" (UID: "51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f"). InnerVolumeSpecName "kube-api-access-lt8kq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:25:12.497444 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.497418 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqc5l"] Jul 23 15:25:12.500964 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:25:12.500941 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f81edca_0ad4_458f_93b0_810787b6e2f5.slice/crio-a2feefdea513597cafc8494794c55eacc8eae80bf552f12d00cf3a43bdbba883 WatchSource:0}: Error finding container a2feefdea513597cafc8494794c55eacc8eae80bf552f12d00cf3a43bdbba883: Status 404 returned error can't find the container with id a2feefdea513597cafc8494794c55eacc8eae80bf552f12d00cf3a43bdbba883 Jul 23 15:25:12.501827 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:25:12.501800 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod263bc01f_e1e1_4329_9c95_2785b4818ede.slice/crio-5e108398ff2bb1d594aa06474560f89ea81516b2749e9b1f830af5023d7b827f WatchSource:0}: Error finding container 5e108398ff2bb1d594aa06474560f89ea81516b2749e9b1f830af5023d7b827f: Status 404 returned error can't find the container with id 5e108398ff2bb1d594aa06474560f89ea81516b2749e9b1f830af5023d7b827f Jul 23 15:25:12.538340 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.538312 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:25:12.540812 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:25:12.540784 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-c6cjh\" is forbidden: User \"system:node:ip-10-0-129-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-63.ec2.internal' and this object" podUID="98445cd4-1ff3-4205-86c0-a608453704dc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:25:12.586386 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.586359 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f-extensions-socket-volume\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:25:12.586386 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.586383 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt8kq\" (UniqueName: \"kubernetes.io/projected/51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f-kube-api-access-lt8kq\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:25:12.686889 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.686862 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdb4j\" (UniqueName: \"kubernetes.io/projected/98445cd4-1ff3-4205-86c0-a608453704dc-kube-api-access-kdb4j\") pod \"98445cd4-1ff3-4205-86c0-a608453704dc\" (UID: \"98445cd4-1ff3-4205-86c0-a608453704dc\") " Jul 23 15:25:12.688824 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.688799 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98445cd4-1ff3-4205-86c0-a608453704dc-kube-api-access-kdb4j" (OuterVolumeSpecName: "kube-api-access-kdb4j") pod "98445cd4-1ff3-4205-86c0-a608453704dc" (UID: "98445cd4-1ff3-4205-86c0-a608453704dc"). InnerVolumeSpecName "kube-api-access-kdb4j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:25:12.730169 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.730135 2570 generic.go:356] "Generic (PLEG): container finished" podID="51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" containerID="98eac5b678c9a7a91da0fb6e1a6d6e9e41016d48d830ee13880ddd06c8de5b70" exitCode=0 Jul 23 15:25:12.730169 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.730174 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:25:12.730421 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.730218 2570 scope.go:117] "RemoveContainer" containerID="98eac5b678c9a7a91da0fb6e1a6d6e9e41016d48d830ee13880ddd06c8de5b70" Jul 23 15:25:12.732007 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.731975 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqc5l" event={"ID":"263bc01f-e1e1-4329-9c95-2785b4818ede","Type":"ContainerStarted","Data":"c684508765326386e9d4598399d96d2d435b0ac67baf7e7652ebc52fd3330902"} Jul 23 15:25:12.732127 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.732019 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqc5l" event={"ID":"263bc01f-e1e1-4329-9c95-2785b4818ede","Type":"ContainerStarted","Data":"5e108398ff2bb1d594aa06474560f89ea81516b2749e9b1f830af5023d7b827f"} Jul 23 15:25:12.732127 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.732060 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqc5l" Jul 23 15:25:12.733088 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:25:12.733062 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fq6ks\" is forbidden: User \"system:node:ip-10-0-129-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-63.ec2.internal' and this object" podUID="51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:25:12.733910 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.733885 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" event={"ID":"9f81edca-0ad4-458f-93b0-810787b6e2f5","Type":"ContainerStarted","Data":"00018f383657c052374158afd4e8c424b838c5cdfc733a87d6db59f716f5da06"} Jul 23 15:25:12.734014 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.733935 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" event={"ID":"9f81edca-0ad4-458f-93b0-810787b6e2f5","Type":"ContainerStarted","Data":"a2feefdea513597cafc8494794c55eacc8eae80bf552f12d00cf3a43bdbba883"} Jul 23 15:25:12.734014 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.733995 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" Jul 23 15:25:12.735772 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:25:12.735647 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-c6cjh\" is forbidden: User \"system:node:ip-10-0-129-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-63.ec2.internal' and this object" podUID="98445cd4-1ff3-4205-86c0-a608453704dc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:25:12.735894 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.735856 2570 generic.go:356] "Generic (PLEG): container finished" podID="98445cd4-1ff3-4205-86c0-a608453704dc" containerID="b0de9d31787a2a489b4d542ba7f2e33faa88c10b81830fc27da4dc8de8dd13bd" exitCode=0 Jul 23 15:25:12.735894 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.735884 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:25:12.737601 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.737578 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" event={"ID":"78cbfd02-7da4-4ecd-8416-fe34927d9c63","Type":"ContainerStarted","Data":"e72dda2d3d6365706a8493f83d282e69d04bcae9d242f12e2720141942f0adb3"} Jul 23 15:25:12.737883 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:25:12.737845 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fq6ks\" is forbidden: User \"system:node:ip-10-0-129-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-63.ec2.internal' and this object" podUID="51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:25:12.741261 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.741127 2570 scope.go:117] "RemoveContainer" containerID="98eac5b678c9a7a91da0fb6e1a6d6e9e41016d48d830ee13880ddd06c8de5b70" Jul 23 15:25:12.741499 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:25:12.741476 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98eac5b678c9a7a91da0fb6e1a6d6e9e41016d48d830ee13880ddd06c8de5b70\": container with ID starting with 98eac5b678c9a7a91da0fb6e1a6d6e9e41016d48d830ee13880ddd06c8de5b70 not found: ID does not exist" containerID="98eac5b678c9a7a91da0fb6e1a6d6e9e41016d48d830ee13880ddd06c8de5b70" Jul 23 15:25:12.741583 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.741509 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98eac5b678c9a7a91da0fb6e1a6d6e9e41016d48d830ee13880ddd06c8de5b70"} err="failed to get container status \"98eac5b678c9a7a91da0fb6e1a6d6e9e41016d48d830ee13880ddd06c8de5b70\": rpc error: code = NotFound desc = could not find container \"98eac5b678c9a7a91da0fb6e1a6d6e9e41016d48d830ee13880ddd06c8de5b70\": container with ID starting with 98eac5b678c9a7a91da0fb6e1a6d6e9e41016d48d830ee13880ddd06c8de5b70 not found: ID does not exist" Jul 23 15:25:12.741583 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.741530 2570 scope.go:117] "RemoveContainer" containerID="b0de9d31787a2a489b4d542ba7f2e33faa88c10b81830fc27da4dc8de8dd13bd" Jul 23 15:25:12.749895 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.749876 2570 scope.go:117] "RemoveContainer" containerID="b0de9d31787a2a489b4d542ba7f2e33faa88c10b81830fc27da4dc8de8dd13bd" Jul 23 15:25:12.750156 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:25:12.750139 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0de9d31787a2a489b4d542ba7f2e33faa88c10b81830fc27da4dc8de8dd13bd\": container with ID starting with b0de9d31787a2a489b4d542ba7f2e33faa88c10b81830fc27da4dc8de8dd13bd not found: ID does not exist" containerID="b0de9d31787a2a489b4d542ba7f2e33faa88c10b81830fc27da4dc8de8dd13bd" Jul 23 15:25:12.750194 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.750164 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0de9d31787a2a489b4d542ba7f2e33faa88c10b81830fc27da4dc8de8dd13bd"} err="failed to get container status \"b0de9d31787a2a489b4d542ba7f2e33faa88c10b81830fc27da4dc8de8dd13bd\": rpc error: code = NotFound desc = could not find container \"b0de9d31787a2a489b4d542ba7f2e33faa88c10b81830fc27da4dc8de8dd13bd\": container with ID starting with b0de9d31787a2a489b4d542ba7f2e33faa88c10b81830fc27da4dc8de8dd13bd not found: ID does not exist" Jul 23 15:25:12.764758 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.763235 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" podStartSLOduration=13.763217483 podStartE2EDuration="13.763217483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:24:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:25:12.760429207 +0000 UTC m=+766.376476472" watchObservedRunningTime="2026-07-23 15:25:12.763217483 +0000 UTC m=+766.379264748" Jul 23 15:25:12.783869 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.783807 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqc5l" podStartSLOduration=13.783792563 podStartE2EDuration="13.783792563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:24:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:25:12.782796676 +0000 UTC m=+766.398843962" watchObservedRunningTime="2026-07-23 15:25:12.783792563 +0000 UTC m=+766.399839826" Jul 23 15:25:12.785028 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:25:12.785001 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-c6cjh\" is forbidden: User \"system:node:ip-10-0-129-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-63.ec2.internal' and this object" podUID="98445cd4-1ff3-4205-86c0-a608453704dc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:25:12.787332 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:25:12.787280 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-c6cjh\" is forbidden: User \"system:node:ip-10-0-129-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-63.ec2.internal' and this object" podUID="98445cd4-1ff3-4205-86c0-a608453704dc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-c6cjh" Jul 23 15:25:12.787597 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.787581 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdb4j\" (UniqueName: \"kubernetes.io/projected/98445cd4-1ff3-4205-86c0-a608453704dc-kube-api-access-kdb4j\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:25:12.789445 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:25:12.789420 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fq6ks\" is forbidden: User \"system:node:ip-10-0-129-63.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-63.ec2.internal' and this object" podUID="51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fq6ks" Jul 23 15:25:12.808364 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:12.808324 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zh7f4" podStartSLOduration=1.834663125 podStartE2EDuration="24.808310071s" totalImagesPullingTime="22.973646946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:24:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:25:12.806422749 +0000 UTC m=+766.422470014" watchObservedRunningTime="2026-07-23 15:25:12.808310071 +0000 UTC m=+766.424357326" Jul 23 15:25:14.438310 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:14.438262 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f" path="/var/lib/kubelet/pods/51ca8b0a-1163-4e13-8e5c-5b6a502a8e0f/volumes" Jul 23 15:25:14.438667 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:14.438604 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98445cd4-1ff3-4205-86c0-a608453704dc" path="/var/lib/kubelet/pods/98445cd4-1ff3-4205-86c0-a608453704dc/volumes" Jul 23 15:25:15.327376 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.327344 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n"] Jul 23 15:25:15.327610 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.327567 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" podUID="9f81edca-0ad4-458f-93b0-810787b6e2f5" containerName="manager" containerID="cri-o://00018f383657c052374158afd4e8c424b838c5cdfc733a87d6db59f716f5da06" gracePeriod=10 Jul 23 15:25:15.641254 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.641227 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" Jul 23 15:25:15.756177 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.756140 2570 generic.go:356] "Generic (PLEG): container finished" podID="9f81edca-0ad4-458f-93b0-810787b6e2f5" containerID="00018f383657c052374158afd4e8c424b838c5cdfc733a87d6db59f716f5da06" exitCode=0 Jul 23 15:25:15.756378 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.756183 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" Jul 23 15:25:15.756378 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.756206 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" event={"ID":"9f81edca-0ad4-458f-93b0-810787b6e2f5","Type":"ContainerDied","Data":"00018f383657c052374158afd4e8c424b838c5cdfc733a87d6db59f716f5da06"} Jul 23 15:25:15.756378 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.756247 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n" event={"ID":"9f81edca-0ad4-458f-93b0-810787b6e2f5","Type":"ContainerDied","Data":"a2feefdea513597cafc8494794c55eacc8eae80bf552f12d00cf3a43bdbba883"} Jul 23 15:25:15.756378 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.756269 2570 scope.go:117] "RemoveContainer" containerID="00018f383657c052374158afd4e8c424b838c5cdfc733a87d6db59f716f5da06" Jul 23 15:25:15.764948 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.764931 2570 scope.go:117] "RemoveContainer" containerID="00018f383657c052374158afd4e8c424b838c5cdfc733a87d6db59f716f5da06" Jul 23 15:25:15.765178 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:25:15.765150 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00018f383657c052374158afd4e8c424b838c5cdfc733a87d6db59f716f5da06\": container with ID starting with 00018f383657c052374158afd4e8c424b838c5cdfc733a87d6db59f716f5da06 not found: ID does not exist" containerID="00018f383657c052374158afd4e8c424b838c5cdfc733a87d6db59f716f5da06" Jul 23 15:25:15.765225 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.765182 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00018f383657c052374158afd4e8c424b838c5cdfc733a87d6db59f716f5da06"} err="failed to get container status \"00018f383657c052374158afd4e8c424b838c5cdfc733a87d6db59f716f5da06\": rpc error: code = NotFound desc = could not find container \"00018f383657c052374158afd4e8c424b838c5cdfc733a87d6db59f716f5da06\": container with ID starting with 00018f383657c052374158afd4e8c424b838c5cdfc733a87d6db59f716f5da06 not found: ID does not exist" Jul 23 15:25:15.815708 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.815676 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f81edca-0ad4-458f-93b0-810787b6e2f5-extensions-socket-volume\") pod \"9f81edca-0ad4-458f-93b0-810787b6e2f5\" (UID: \"9f81edca-0ad4-458f-93b0-810787b6e2f5\") " Jul 23 15:25:15.815875 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.815743 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvd7m\" (UniqueName: \"kubernetes.io/projected/9f81edca-0ad4-458f-93b0-810787b6e2f5-kube-api-access-gvd7m\") pod \"9f81edca-0ad4-458f-93b0-810787b6e2f5\" (UID: \"9f81edca-0ad4-458f-93b0-810787b6e2f5\") " Jul 23 15:25:15.816077 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.816054 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f81edca-0ad4-458f-93b0-810787b6e2f5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9f81edca-0ad4-458f-93b0-810787b6e2f5" (UID: "9f81edca-0ad4-458f-93b0-810787b6e2f5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:25:15.817757 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.817736 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f81edca-0ad4-458f-93b0-810787b6e2f5-kube-api-access-gvd7m" (OuterVolumeSpecName: "kube-api-access-gvd7m") pod "9f81edca-0ad4-458f-93b0-810787b6e2f5" (UID: "9f81edca-0ad4-458f-93b0-810787b6e2f5"). InnerVolumeSpecName "kube-api-access-gvd7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:25:15.916438 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.916400 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f81edca-0ad4-458f-93b0-810787b6e2f5-extensions-socket-volume\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:25:15.916438 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:15.916431 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvd7m\" (UniqueName: \"kubernetes.io/projected/9f81edca-0ad4-458f-93b0-810787b6e2f5-kube-api-access-gvd7m\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:25:16.095642 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:16.095606 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n"] Jul 23 15:25:16.100497 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:16.100472 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-txt8n"] Jul 23 15:25:16.439180 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:16.439150 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f81edca-0ad4-458f-93b0-810787b6e2f5" path="/var/lib/kubelet/pods/9f81edca-0ad4-458f-93b0-810787b6e2f5/volumes" Jul 23 15:25:23.747878 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:23.747841 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bqc5l" Jul 23 15:25:31.779992 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.779956 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5"] Jul 23 15:25:31.780393 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.780385 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f81edca-0ad4-458f-93b0-810787b6e2f5" containerName="manager" Jul 23 15:25:31.780441 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.780397 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f81edca-0ad4-458f-93b0-810787b6e2f5" containerName="manager" Jul 23 15:25:31.780485 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.780474 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f81edca-0ad4-458f-93b0-810787b6e2f5" containerName="manager" Jul 23 15:25:31.783208 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.783188 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.787620 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.787592 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-hf2fv\"" Jul 23 15:25:31.828861 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.828816 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5"] Jul 23 15:25:31.846731 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.846698 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.846880 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.846747 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.846880 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.846778 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.846880 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.846802 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.846880 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.846818 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j8dv\" (UniqueName: \"kubernetes.io/projected/bb897d59-bcb8-437a-8a31-706f28a1e6e6-kube-api-access-4j8dv\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.846880 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.846839 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.847043 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.846882 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.847043 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.846903 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.847043 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.846992 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.948142 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.948098 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.948142 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.948146 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.948142 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.948166 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.948472 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.948186 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j8dv\" (UniqueName: \"kubernetes.io/projected/bb897d59-bcb8-437a-8a31-706f28a1e6e6-kube-api-access-4j8dv\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.948472 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.948219 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.948472 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.948252 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.948472 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.948278 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.948472 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.948377 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.948472 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.948417 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.948789 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.948756 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.948911 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.948831 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.948911 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.948873 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.948911 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.948882 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.949288 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.949248 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.951145 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.951123 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.951416 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.951401 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.964919 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.964894 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j8dv\" (UniqueName: \"kubernetes.io/projected/bb897d59-bcb8-437a-8a31-706f28a1e6e6-kube-api-access-4j8dv\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:31.966507 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:31.966485 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb897d59-bcb8-437a-8a31-706f28a1e6e6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-h5kf5\" (UID: \"bb897d59-bcb8-437a-8a31-706f28a1e6e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:32.093123 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:32.093031 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:32.229961 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:32.229938 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5"] Jul 23 15:25:32.231933 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:25:32.231909 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb897d59_bcb8_437a_8a31_706f28a1e6e6.slice/crio-42f469f430da00a4ec37f5d062b449e28892602345b190503bbc336e91c7b3fc WatchSource:0}: Error finding container 42f469f430da00a4ec37f5d062b449e28892602345b190503bbc336e91c7b3fc: Status 404 returned error can't find the container with id 42f469f430da00a4ec37f5d062b449e28892602345b190503bbc336e91c7b3fc Jul 23 15:25:32.234101 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:32.234068 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:25:32.234187 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:32.234134 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:25:32.234233 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:32.234185 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:25:32.838352 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:32.838290 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" event={"ID":"bb897d59-bcb8-437a-8a31-706f28a1e6e6","Type":"ContainerStarted","Data":"7b2e87b17c822cfbf27025451456fc116f16090ceb991d49929dc26cbcdd9ed2"} Jul 23 15:25:32.838734 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:32.838370 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" event={"ID":"bb897d59-bcb8-437a-8a31-706f28a1e6e6","Type":"ContainerStarted","Data":"42f469f430da00a4ec37f5d062b449e28892602345b190503bbc336e91c7b3fc"} Jul 23 15:25:32.867706 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:32.867661 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" podStartSLOduration=1.8676464369999999 podStartE2EDuration="1.867646437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:25:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:25:32.865665599 +0000 UTC m=+786.481712862" watchObservedRunningTime="2026-07-23 15:25:32.867646437 +0000 UTC m=+786.483693700" Jul 23 15:25:33.093234 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:33.093147 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:33.095037 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:33.095013 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-h5kf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 23 15:25:33.095160 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:33.095062 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" podUID="bb897d59-bcb8-437a-8a31-706f28a1e6e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 23 15:25:34.097781 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:34.097745 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:34.847183 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:34.847153 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:34.847951 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:34.847930 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:25:44.280269 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:44.280227 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lm477"] Jul 23 15:25:44.352625 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:44.352589 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lm477"] Jul 23 15:25:44.352792 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:44.352716 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" Jul 23 15:25:44.360755 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:44.360726 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 23 15:25:44.457580 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:44.457550 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxntn\" (UniqueName: \"kubernetes.io/projected/8287141d-b73e-4fac-b0ce-85ea4b44ef88-kube-api-access-mxntn\") pod \"limitador-limitador-64c94fd64f-lm477\" (UID: \"8287141d-b73e-4fac-b0ce-85ea4b44ef88\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" Jul 23 15:25:44.457754 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:44.457606 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8287141d-b73e-4fac-b0ce-85ea4b44ef88-config-file\") pod \"limitador-limitador-64c94fd64f-lm477\" (UID: \"8287141d-b73e-4fac-b0ce-85ea4b44ef88\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" Jul 23 15:25:44.473906 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:44.473876 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lm477"] Jul 23 15:25:44.558514 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:44.558438 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxntn\" (UniqueName: \"kubernetes.io/projected/8287141d-b73e-4fac-b0ce-85ea4b44ef88-kube-api-access-mxntn\") pod \"limitador-limitador-64c94fd64f-lm477\" (UID: \"8287141d-b73e-4fac-b0ce-85ea4b44ef88\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" Jul 23 15:25:44.558514 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:44.558475 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8287141d-b73e-4fac-b0ce-85ea4b44ef88-config-file\") pod \"limitador-limitador-64c94fd64f-lm477\" (UID: \"8287141d-b73e-4fac-b0ce-85ea4b44ef88\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" Jul 23 15:25:44.559106 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:44.559088 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8287141d-b73e-4fac-b0ce-85ea4b44ef88-config-file\") pod \"limitador-limitador-64c94fd64f-lm477\" (UID: \"8287141d-b73e-4fac-b0ce-85ea4b44ef88\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" Jul 23 15:25:44.568355 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:44.568319 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxntn\" (UniqueName: \"kubernetes.io/projected/8287141d-b73e-4fac-b0ce-85ea4b44ef88-kube-api-access-mxntn\") pod \"limitador-limitador-64c94fd64f-lm477\" (UID: \"8287141d-b73e-4fac-b0ce-85ea4b44ef88\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" Jul 23 15:25:44.664104 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:44.664070 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" Jul 23 15:25:44.796171 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:44.796146 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lm477"] Jul 23 15:25:44.798614 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:25:44.798586 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8287141d_b73e_4fac_b0ce_85ea4b44ef88.slice/crio-d4e3a5c60c84ccf5ca2e807c912c1c74224b6bcb7e65cac1b648c79bcaf99cbe WatchSource:0}: Error finding container d4e3a5c60c84ccf5ca2e807c912c1c74224b6bcb7e65cac1b648c79bcaf99cbe: Status 404 returned error can't find the container with id d4e3a5c60c84ccf5ca2e807c912c1c74224b6bcb7e65cac1b648c79bcaf99cbe Jul 23 15:25:44.894982 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:44.894948 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" event={"ID":"8287141d-b73e-4fac-b0ce-85ea4b44ef88","Type":"ContainerStarted","Data":"d4e3a5c60c84ccf5ca2e807c912c1c74224b6bcb7e65cac1b648c79bcaf99cbe"} Jul 23 15:25:45.233409 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:45.233326 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-mjhgb"] Jul 23 15:25:45.238150 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:45.238130 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mjhgb" Jul 23 15:25:45.240774 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:45.240754 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6s9b8\"" Jul 23 15:25:45.244491 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:45.244462 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mjhgb"] Jul 23 15:25:45.366195 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:45.366163 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbhbv\" (UniqueName: \"kubernetes.io/projected/49c4921b-285a-405b-8c5f-85c02fc68d89-kube-api-access-dbhbv\") pod \"authorino-58cd955675-mjhgb\" (UID: \"49c4921b-285a-405b-8c5f-85c02fc68d89\") " pod="kuadrant-system/authorino-58cd955675-mjhgb" Jul 23 15:25:45.466863 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:45.466828 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbhbv\" (UniqueName: \"kubernetes.io/projected/49c4921b-285a-405b-8c5f-85c02fc68d89-kube-api-access-dbhbv\") pod \"authorino-58cd955675-mjhgb\" (UID: \"49c4921b-285a-405b-8c5f-85c02fc68d89\") " pod="kuadrant-system/authorino-58cd955675-mjhgb" Jul 23 15:25:45.477091 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:45.477067 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbhbv\" (UniqueName: \"kubernetes.io/projected/49c4921b-285a-405b-8c5f-85c02fc68d89-kube-api-access-dbhbv\") pod \"authorino-58cd955675-mjhgb\" (UID: \"49c4921b-285a-405b-8c5f-85c02fc68d89\") " pod="kuadrant-system/authorino-58cd955675-mjhgb" Jul 23 15:25:45.548004 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:45.547924 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mjhgb" Jul 23 15:25:45.944528 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:45.944454 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mjhgb"] Jul 23 15:25:45.961075 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:25:45.956138 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49c4921b_285a_405b_8c5f_85c02fc68d89.slice/crio-e087c03e135ab583fcb983e5eefa6f1553b3c5a1117037ea0fea279f90d25f30 WatchSource:0}: Error finding container e087c03e135ab583fcb983e5eefa6f1553b3c5a1117037ea0fea279f90d25f30: Status 404 returned error can't find the container with id e087c03e135ab583fcb983e5eefa6f1553b3c5a1117037ea0fea279f90d25f30 Jul 23 15:25:46.909950 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:46.909909 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mjhgb" event={"ID":"49c4921b-285a-405b-8c5f-85c02fc68d89","Type":"ContainerStarted","Data":"e087c03e135ab583fcb983e5eefa6f1553b3c5a1117037ea0fea279f90d25f30"} Jul 23 15:25:47.919225 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:47.919174 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" event={"ID":"8287141d-b73e-4fac-b0ce-85ea4b44ef88","Type":"ContainerStarted","Data":"c5534650821e965c8f300ef41e869cfe8b52071d0850f332a2ad0ad1614ed9a1"} Jul 23 15:25:47.919704 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:47.919337 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" Jul 23 15:25:47.943558 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:47.943508 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" podStartSLOduration=0.990228546 podStartE2EDuration="3.943491149s" totalImagesPullingTime="2.953262603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:25:47.939840236 +0000 UTC m=+801.555887537" watchObservedRunningTime="2026-07-23 15:25:47.943491149 +0000 UTC m=+801.559538413" Jul 23 15:25:49.516545 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:49.516456 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mjhgb"] Jul 23 15:25:49.930859 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:49.930820 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mjhgb" event={"ID":"49c4921b-285a-405b-8c5f-85c02fc68d89","Type":"ContainerStarted","Data":"2010d406aba06347ea6a5a155b82e1305cf9d1c58da94073d05522ad299a8736"} Jul 23 15:25:49.931052 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:49.930830 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-mjhgb" podUID="49c4921b-285a-405b-8c5f-85c02fc68d89" containerName="authorino" containerID="cri-o://2010d406aba06347ea6a5a155b82e1305cf9d1c58da94073d05522ad299a8736" gracePeriod=30 Jul 23 15:25:49.948200 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:49.948158 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-mjhgb" podStartSLOduration=1.246052561 podStartE2EDuration="4.948142431s" totalImagesPullingTime="3.70208987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:25:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:25:49.945901209 +0000 UTC m=+803.561948470" watchObservedRunningTime="2026-07-23 15:25:49.948142431 +0000 UTC m=+803.564189695" Jul 23 15:25:50.184472 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:50.184413 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mjhgb" Jul 23 15:25:50.313818 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:50.313784 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbhbv\" (UniqueName: \"kubernetes.io/projected/49c4921b-285a-405b-8c5f-85c02fc68d89-kube-api-access-dbhbv\") pod \"49c4921b-285a-405b-8c5f-85c02fc68d89\" (UID: \"49c4921b-285a-405b-8c5f-85c02fc68d89\") " Jul 23 15:25:50.316057 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:50.316025 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49c4921b-285a-405b-8c5f-85c02fc68d89-kube-api-access-dbhbv" (OuterVolumeSpecName: "kube-api-access-dbhbv") pod "49c4921b-285a-405b-8c5f-85c02fc68d89" (UID: "49c4921b-285a-405b-8c5f-85c02fc68d89"). InnerVolumeSpecName "kube-api-access-dbhbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:25:50.415090 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:50.415062 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbhbv\" (UniqueName: \"kubernetes.io/projected/49c4921b-285a-405b-8c5f-85c02fc68d89-kube-api-access-dbhbv\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:25:50.936676 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:50.936640 2570 generic.go:356] "Generic (PLEG): container finished" podID="49c4921b-285a-405b-8c5f-85c02fc68d89" containerID="2010d406aba06347ea6a5a155b82e1305cf9d1c58da94073d05522ad299a8736" exitCode=0 Jul 23 15:25:50.937137 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:50.936704 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mjhgb" Jul 23 15:25:50.937137 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:50.936702 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mjhgb" event={"ID":"49c4921b-285a-405b-8c5f-85c02fc68d89","Type":"ContainerDied","Data":"2010d406aba06347ea6a5a155b82e1305cf9d1c58da94073d05522ad299a8736"} Jul 23 15:25:50.937137 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:50.936821 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mjhgb" event={"ID":"49c4921b-285a-405b-8c5f-85c02fc68d89","Type":"ContainerDied","Data":"e087c03e135ab583fcb983e5eefa6f1553b3c5a1117037ea0fea279f90d25f30"} Jul 23 15:25:50.937137 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:50.936848 2570 scope.go:117] "RemoveContainer" containerID="2010d406aba06347ea6a5a155b82e1305cf9d1c58da94073d05522ad299a8736" Jul 23 15:25:50.945216 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:50.945188 2570 scope.go:117] "RemoveContainer" containerID="2010d406aba06347ea6a5a155b82e1305cf9d1c58da94073d05522ad299a8736" Jul 23 15:25:50.945490 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:25:50.945470 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2010d406aba06347ea6a5a155b82e1305cf9d1c58da94073d05522ad299a8736\": container with ID starting with 2010d406aba06347ea6a5a155b82e1305cf9d1c58da94073d05522ad299a8736 not found: ID does not exist" containerID="2010d406aba06347ea6a5a155b82e1305cf9d1c58da94073d05522ad299a8736" Jul 23 15:25:50.945558 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:50.945499 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2010d406aba06347ea6a5a155b82e1305cf9d1c58da94073d05522ad299a8736"} err="failed to get container status \"2010d406aba06347ea6a5a155b82e1305cf9d1c58da94073d05522ad299a8736\": rpc error: code = NotFound desc = could not find container \"2010d406aba06347ea6a5a155b82e1305cf9d1c58da94073d05522ad299a8736\": container with ID starting with 2010d406aba06347ea6a5a155b82e1305cf9d1c58da94073d05522ad299a8736 not found: ID does not exist" Jul 23 15:25:50.958213 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:50.958191 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mjhgb"] Jul 23 15:25:50.962664 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:50.962642 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-mjhgb"] Jul 23 15:25:52.438548 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:52.438508 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49c4921b-285a-405b-8c5f-85c02fc68d89" path="/var/lib/kubelet/pods/49c4921b-285a-405b-8c5f-85c02fc68d89/volumes" Jul 23 15:25:58.926127 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:58.926093 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" Jul 23 15:25:59.308661 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:59.308571 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lm477"] Jul 23 15:25:59.308840 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:59.308809 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" podUID="8287141d-b73e-4fac-b0ce-85ea4b44ef88" containerName="limitador" containerID="cri-o://c5534650821e965c8f300ef41e869cfe8b52071d0850f332a2ad0ad1614ed9a1" gracePeriod=30 Jul 23 15:25:59.979535 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:59.979448 2570 generic.go:356] "Generic (PLEG): container finished" podID="8287141d-b73e-4fac-b0ce-85ea4b44ef88" containerID="c5534650821e965c8f300ef41e869cfe8b52071d0850f332a2ad0ad1614ed9a1" exitCode=0 Jul 23 15:25:59.979535 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:25:59.979521 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" event={"ID":"8287141d-b73e-4fac-b0ce-85ea4b44ef88","Type":"ContainerDied","Data":"c5534650821e965c8f300ef41e869cfe8b52071d0850f332a2ad0ad1614ed9a1"} Jul 23 15:26:00.262169 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:00.262143 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" Jul 23 15:26:00.402184 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:00.402158 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8287141d-b73e-4fac-b0ce-85ea4b44ef88-config-file\") pod \"8287141d-b73e-4fac-b0ce-85ea4b44ef88\" (UID: \"8287141d-b73e-4fac-b0ce-85ea4b44ef88\") " Jul 23 15:26:00.402372 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:00.402265 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxntn\" (UniqueName: \"kubernetes.io/projected/8287141d-b73e-4fac-b0ce-85ea4b44ef88-kube-api-access-mxntn\") pod \"8287141d-b73e-4fac-b0ce-85ea4b44ef88\" (UID: \"8287141d-b73e-4fac-b0ce-85ea4b44ef88\") " Jul 23 15:26:00.402504 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:00.402483 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8287141d-b73e-4fac-b0ce-85ea4b44ef88-config-file" (OuterVolumeSpecName: "config-file") pod "8287141d-b73e-4fac-b0ce-85ea4b44ef88" (UID: "8287141d-b73e-4fac-b0ce-85ea4b44ef88"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:26:00.404335 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:00.404284 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8287141d-b73e-4fac-b0ce-85ea4b44ef88-kube-api-access-mxntn" (OuterVolumeSpecName: "kube-api-access-mxntn") pod "8287141d-b73e-4fac-b0ce-85ea4b44ef88" (UID: "8287141d-b73e-4fac-b0ce-85ea4b44ef88"). InnerVolumeSpecName "kube-api-access-mxntn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:26:00.503255 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:00.503191 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxntn\" (UniqueName: \"kubernetes.io/projected/8287141d-b73e-4fac-b0ce-85ea4b44ef88-kube-api-access-mxntn\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:26:00.503255 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:00.503213 2570 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8287141d-b73e-4fac-b0ce-85ea4b44ef88-config-file\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:26:00.985476 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:00.985440 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" event={"ID":"8287141d-b73e-4fac-b0ce-85ea4b44ef88","Type":"ContainerDied","Data":"d4e3a5c60c84ccf5ca2e807c912c1c74224b6bcb7e65cac1b648c79bcaf99cbe"} Jul 23 15:26:00.985476 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:00.985464 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-lm477" Jul 23 15:26:00.986026 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:00.985498 2570 scope.go:117] "RemoveContainer" containerID="c5534650821e965c8f300ef41e869cfe8b52071d0850f332a2ad0ad1614ed9a1" Jul 23 15:26:01.006477 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:01.006434 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lm477"] Jul 23 15:26:01.008517 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:01.008499 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-lm477"] Jul 23 15:26:02.442267 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:02.442227 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8287141d-b73e-4fac-b0ce-85ea4b44ef88" path="/var/lib/kubelet/pods/8287141d-b73e-4fac-b0ce-85ea4b44ef88/volumes" Jul 23 15:26:05.758742 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.758704 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx"] Jul 23 15:26:05.759230 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.759212 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8287141d-b73e-4fac-b0ce-85ea4b44ef88" containerName="limitador" Jul 23 15:26:05.759344 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.759233 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8287141d-b73e-4fac-b0ce-85ea4b44ef88" containerName="limitador" Jul 23 15:26:05.759344 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.759250 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49c4921b-285a-405b-8c5f-85c02fc68d89" containerName="authorino" Jul 23 15:26:05.759344 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.759259 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="49c4921b-285a-405b-8c5f-85c02fc68d89" containerName="authorino" Jul 23 15:26:05.759498 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.759412 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="49c4921b-285a-405b-8c5f-85c02fc68d89" containerName="authorino" Jul 23 15:26:05.759498 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.759429 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8287141d-b73e-4fac-b0ce-85ea4b44ef88" containerName="limitador" Jul 23 15:26:05.762376 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.762355 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx" Jul 23 15:26:05.765528 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.765496 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-qkdpz\"" Jul 23 15:26:05.765648 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.765508 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 23 15:26:05.765648 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.765528 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 15:26:05.766435 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.766420 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 23 15:26:05.771829 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.771809 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx"] Jul 23 15:26:05.850813 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.850784 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6d34fa9a-0b4f-409a-9f2a-d99f3c69042d-data\") pod \"postgres-69fb4594cf-gtvcx\" (UID: \"6d34fa9a-0b4f-409a-9f2a-d99f3c69042d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx" Jul 23 15:26:05.851002 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.850835 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tqzg\" (UniqueName: \"kubernetes.io/projected/6d34fa9a-0b4f-409a-9f2a-d99f3c69042d-kube-api-access-2tqzg\") pod \"postgres-69fb4594cf-gtvcx\" (UID: \"6d34fa9a-0b4f-409a-9f2a-d99f3c69042d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx" Jul 23 15:26:05.951456 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.951418 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6d34fa9a-0b4f-409a-9f2a-d99f3c69042d-data\") pod \"postgres-69fb4594cf-gtvcx\" (UID: \"6d34fa9a-0b4f-409a-9f2a-d99f3c69042d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx" Jul 23 15:26:05.951648 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.951496 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tqzg\" (UniqueName: \"kubernetes.io/projected/6d34fa9a-0b4f-409a-9f2a-d99f3c69042d-kube-api-access-2tqzg\") pod \"postgres-69fb4594cf-gtvcx\" (UID: \"6d34fa9a-0b4f-409a-9f2a-d99f3c69042d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx" Jul 23 15:26:05.951824 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.951806 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6d34fa9a-0b4f-409a-9f2a-d99f3c69042d-data\") pod \"postgres-69fb4594cf-gtvcx\" (UID: \"6d34fa9a-0b4f-409a-9f2a-d99f3c69042d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx" Jul 23 15:26:05.961050 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:05.961019 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tqzg\" (UniqueName: \"kubernetes.io/projected/6d34fa9a-0b4f-409a-9f2a-d99f3c69042d-kube-api-access-2tqzg\") pod \"postgres-69fb4594cf-gtvcx\" (UID: \"6d34fa9a-0b4f-409a-9f2a-d99f3c69042d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx" Jul 23 15:26:06.073746 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:06.073666 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx" Jul 23 15:26:06.238012 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:06.237987 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx"] Jul 23 15:26:06.239688 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:26:06.239660 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d34fa9a_0b4f_409a_9f2a_d99f3c69042d.slice/crio-80e24f354d76a81fae1093f8ca57740b0d6ddd7da30ddabda1a4acf338db39be WatchSource:0}: Error finding container 80e24f354d76a81fae1093f8ca57740b0d6ddd7da30ddabda1a4acf338db39be: Status 404 returned error can't find the container with id 80e24f354d76a81fae1093f8ca57740b0d6ddd7da30ddabda1a4acf338db39be Jul 23 15:26:07.018382 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:07.018283 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx" event={"ID":"6d34fa9a-0b4f-409a-9f2a-d99f3c69042d","Type":"ContainerStarted","Data":"80e24f354d76a81fae1093f8ca57740b0d6ddd7da30ddabda1a4acf338db39be"} Jul 23 15:26:11.048716 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:11.048680 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx" event={"ID":"6d34fa9a-0b4f-409a-9f2a-d99f3c69042d","Type":"ContainerStarted","Data":"544893f5dda8dc5fee6dc2a65e6da382d8f4ba23254b60fbd32ea973e6b5d11b"} Jul 23 15:26:11.049060 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:11.048783 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx" Jul 23 15:26:11.065868 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:11.065815 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx" podStartSLOduration=1.480329759 podStartE2EDuration="6.065798956s" totalImagesPullingTime="4.585469197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:26:11.064008421 +0000 UTC m=+824.680055685" watchObservedRunningTime="2026-07-23 15:26:11.065798956 +0000 UTC m=+824.681846249" Jul 23 15:26:17.084361 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:17.084332 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-gtvcx" Jul 23 15:26:17.685871 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:17.685839 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w9j6v"] Jul 23 15:26:17.690693 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:17.690673 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-w9j6v" Jul 23 15:26:17.705065 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:17.705043 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6s9b8\"" Jul 23 15:26:17.721763 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:17.721740 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w9j6v"] Jul 23 15:26:17.864608 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:17.864563 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc5xj\" (UniqueName: \"kubernetes.io/projected/69bb0e90-0b3f-4da6-845c-45a3c95c0453-kube-api-access-pc5xj\") pod \"authorino-5d4b67c446-w9j6v\" (UID: \"69bb0e90-0b3f-4da6-845c-45a3c95c0453\") " pod="kuadrant-system/authorino-5d4b67c446-w9j6v" Jul 23 15:26:17.915914 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:17.915865 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w9j6v"] Jul 23 15:26:17.916171 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:26:17.916151 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-pc5xj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-w9j6v" podUID="69bb0e90-0b3f-4da6-845c-45a3c95c0453" Jul 23 15:26:17.958323 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:17.958239 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5549598b75-v5zd4"] Jul 23 15:26:17.961681 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:17.961664 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5549598b75-v5zd4" Jul 23 15:26:17.965585 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:17.965562 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc5xj\" (UniqueName: \"kubernetes.io/projected/69bb0e90-0b3f-4da6-845c-45a3c95c0453-kube-api-access-pc5xj\") pod \"authorino-5d4b67c446-w9j6v\" (UID: \"69bb0e90-0b3f-4da6-845c-45a3c95c0453\") " pod="kuadrant-system/authorino-5d4b67c446-w9j6v" Jul 23 15:26:17.975830 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:17.975808 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5549598b75-v5zd4"] Jul 23 15:26:17.978189 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:17.978168 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc5xj\" (UniqueName: \"kubernetes.io/projected/69bb0e90-0b3f-4da6-845c-45a3c95c0453-kube-api-access-pc5xj\") pod \"authorino-5d4b67c446-w9j6v\" (UID: \"69bb0e90-0b3f-4da6-845c-45a3c95c0453\") " pod="kuadrant-system/authorino-5d4b67c446-w9j6v" Jul 23 15:26:18.067047 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:18.067012 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdmkn\" (UniqueName: \"kubernetes.io/projected/2ed12e58-4061-421a-b50e-a3ff67f60c9a-kube-api-access-rdmkn\") pod \"authorino-5549598b75-v5zd4\" (UID: \"2ed12e58-4061-421a-b50e-a3ff67f60c9a\") " pod="kuadrant-system/authorino-5549598b75-v5zd4" Jul 23 15:26:18.083688 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:18.083662 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-w9j6v" Jul 23 15:26:18.091010 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:18.090990 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-w9j6v" Jul 23 15:26:18.156504 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:18.156471 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5549598b75-v5zd4"] Jul 23 15:26:18.156749 ip-10-0-129-63 kubenswrapper[2570]: E0723 15:26:18.156732 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rdmkn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5549598b75-v5zd4" podUID="2ed12e58-4061-421a-b50e-a3ff67f60c9a" Jul 23 15:26:18.167922 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:18.167891 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdmkn\" (UniqueName: \"kubernetes.io/projected/2ed12e58-4061-421a-b50e-a3ff67f60c9a-kube-api-access-rdmkn\") pod \"authorino-5549598b75-v5zd4\" (UID: \"2ed12e58-4061-421a-b50e-a3ff67f60c9a\") " pod="kuadrant-system/authorino-5549598b75-v5zd4" Jul 23 15:26:18.182847 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:18.182815 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdmkn\" (UniqueName: \"kubernetes.io/projected/2ed12e58-4061-421a-b50e-a3ff67f60c9a-kube-api-access-rdmkn\") pod \"authorino-5549598b75-v5zd4\" (UID: \"2ed12e58-4061-421a-b50e-a3ff67f60c9a\") " pod="kuadrant-system/authorino-5549598b75-v5zd4" Jul 23 15:26:18.268967 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:18.268884 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pc5xj\" (UniqueName: \"kubernetes.io/projected/69bb0e90-0b3f-4da6-845c-45a3c95c0453-kube-api-access-pc5xj\") pod \"69bb0e90-0b3f-4da6-845c-45a3c95c0453\" (UID: \"69bb0e90-0b3f-4da6-845c-45a3c95c0453\") " Jul 23 15:26:18.271065 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:18.271041 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69bb0e90-0b3f-4da6-845c-45a3c95c0453-kube-api-access-pc5xj" (OuterVolumeSpecName: "kube-api-access-pc5xj") pod "69bb0e90-0b3f-4da6-845c-45a3c95c0453" (UID: "69bb0e90-0b3f-4da6-845c-45a3c95c0453"). InnerVolumeSpecName "kube-api-access-pc5xj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:26:18.369582 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:18.369536 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc5xj\" (UniqueName: \"kubernetes.io/projected/69bb0e90-0b3f-4da6-845c-45a3c95c0453-kube-api-access-pc5xj\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:26:19.088056 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:19.088023 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5549598b75-v5zd4" Jul 23 15:26:19.088231 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:19.088026 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-w9j6v" Jul 23 15:26:19.095584 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:19.095559 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5549598b75-v5zd4" Jul 23 15:26:19.175508 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:19.175479 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdmkn\" (UniqueName: \"kubernetes.io/projected/2ed12e58-4061-421a-b50e-a3ff67f60c9a-kube-api-access-rdmkn\") pod \"2ed12e58-4061-421a-b50e-a3ff67f60c9a\" (UID: \"2ed12e58-4061-421a-b50e-a3ff67f60c9a\") " Jul 23 15:26:19.176180 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:19.176155 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w9j6v"] Jul 23 15:26:19.177567 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:19.177542 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ed12e58-4061-421a-b50e-a3ff67f60c9a-kube-api-access-rdmkn" (OuterVolumeSpecName: "kube-api-access-rdmkn") pod "2ed12e58-4061-421a-b50e-a3ff67f60c9a" (UID: "2ed12e58-4061-421a-b50e-a3ff67f60c9a"). InnerVolumeSpecName "kube-api-access-rdmkn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:26:19.182491 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:19.182470 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-w9j6v"] Jul 23 15:26:19.276070 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:19.276030 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdmkn\" (UniqueName: \"kubernetes.io/projected/2ed12e58-4061-421a-b50e-a3ff67f60c9a-kube-api-access-rdmkn\") on node \"ip-10-0-129-63.ec2.internal\" DevicePath \"\"" Jul 23 15:26:20.092440 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:20.092407 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5549598b75-v5zd4" Jul 23 15:26:20.123184 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:20.123154 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5549598b75-v5zd4"] Jul 23 15:26:20.129898 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:20.129872 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5549598b75-v5zd4"] Jul 23 15:26:20.439304 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:20.439251 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ed12e58-4061-421a-b50e-a3ff67f60c9a" path="/var/lib/kubelet/pods/2ed12e58-4061-421a-b50e-a3ff67f60c9a/volumes" Jul 23 15:26:20.439535 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:20.439522 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69bb0e90-0b3f-4da6-845c-45a3c95c0453" path="/var/lib/kubelet/pods/69bb0e90-0b3f-4da6-845c-45a3c95c0453/volumes" Jul 23 15:26:24.527053 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:24.527020 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-847795f4d4-8gz9z"] Jul 23 15:26:24.634822 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:24.634782 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-847795f4d4-8gz9z"] Jul 23 15:26:24.634990 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:24.634831 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-847795f4d4-8gz9z" Jul 23 15:26:24.637829 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:24.637805 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 23 15:26:24.637959 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:24.637835 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 23 15:26:24.638059 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:24.638037 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9zv5m\"" Jul 23 15:26:24.731425 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:24.731385 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95da00bc-38f0-4946-8a7a-abb805f693ec-webhook-cert\") pod \"maas-controller-847795f4d4-8gz9z\" (UID: \"95da00bc-38f0-4946-8a7a-abb805f693ec\") " pod="opendatahub/maas-controller-847795f4d4-8gz9z" Jul 23 15:26:24.731425 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:24.731426 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqpwd\" (UniqueName: \"kubernetes.io/projected/95da00bc-38f0-4946-8a7a-abb805f693ec-kube-api-access-nqpwd\") pod \"maas-controller-847795f4d4-8gz9z\" (UID: \"95da00bc-38f0-4946-8a7a-abb805f693ec\") " pod="opendatahub/maas-controller-847795f4d4-8gz9z" Jul 23 15:26:24.832940 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:24.832853 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqpwd\" (UniqueName: \"kubernetes.io/projected/95da00bc-38f0-4946-8a7a-abb805f693ec-kube-api-access-nqpwd\") pod \"maas-controller-847795f4d4-8gz9z\" (UID: \"95da00bc-38f0-4946-8a7a-abb805f693ec\") " pod="opendatahub/maas-controller-847795f4d4-8gz9z" Jul 23 15:26:24.833104 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:24.833011 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95da00bc-38f0-4946-8a7a-abb805f693ec-webhook-cert\") pod \"maas-controller-847795f4d4-8gz9z\" (UID: \"95da00bc-38f0-4946-8a7a-abb805f693ec\") " pod="opendatahub/maas-controller-847795f4d4-8gz9z" Jul 23 15:26:24.835502 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:24.835480 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/95da00bc-38f0-4946-8a7a-abb805f693ec-webhook-cert\") pod \"maas-controller-847795f4d4-8gz9z\" (UID: \"95da00bc-38f0-4946-8a7a-abb805f693ec\") " pod="opendatahub/maas-controller-847795f4d4-8gz9z" Jul 23 15:26:24.843255 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:24.843231 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqpwd\" (UniqueName: \"kubernetes.io/projected/95da00bc-38f0-4946-8a7a-abb805f693ec-kube-api-access-nqpwd\") pod \"maas-controller-847795f4d4-8gz9z\" (UID: \"95da00bc-38f0-4946-8a7a-abb805f693ec\") " pod="opendatahub/maas-controller-847795f4d4-8gz9z" Jul 23 15:26:24.951007 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:24.950973 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-847795f4d4-8gz9z" Jul 23 15:26:25.110695 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:25.110671 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-847795f4d4-8gz9z"] Jul 23 15:26:25.113289 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:26:25.113264 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95da00bc_38f0_4946_8a7a_abb805f693ec.slice/crio-39837298dceb1e1c1d71e8458c5c09def85170b55fae8eb485c50e1b3c5550de WatchSource:0}: Error finding container 39837298dceb1e1c1d71e8458c5c09def85170b55fae8eb485c50e1b3c5550de: Status 404 returned error can't find the container with id 39837298dceb1e1c1d71e8458c5c09def85170b55fae8eb485c50e1b3c5550de Jul 23 15:26:26.124024 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:26.123942 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-847795f4d4-8gz9z" event={"ID":"95da00bc-38f0-4946-8a7a-abb805f693ec","Type":"ContainerStarted","Data":"39837298dceb1e1c1d71e8458c5c09def85170b55fae8eb485c50e1b3c5550de"} Jul 23 15:26:28.134875 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:28.134838 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-847795f4d4-8gz9z" event={"ID":"95da00bc-38f0-4946-8a7a-abb805f693ec","Type":"ContainerStarted","Data":"28d275c2b526b2fea873351325834c1529443251f7dbc7c11c9fb255368cfb84"} Jul 23 15:26:28.135220 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:28.134944 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-847795f4d4-8gz9z" Jul 23 15:26:39.142069 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:39.142035 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-847795f4d4-8gz9z" Jul 23 15:26:39.162003 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:26:39.161949 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-847795f4d4-8gz9z" podStartSLOduration=12.771288973 podStartE2EDuration="15.161931997s" totalImagesPullingTime="2.390643024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:26:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:26:28.164769091 +0000 UTC m=+841.780816356" watchObservedRunningTime="2026-07-23 15:26:39.161931997 +0000 UTC m=+852.777979263" Jul 23 15:27:01.015104 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.015064 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-lp2j9"] Jul 23 15:27:01.018563 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.018539 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-lp2j9" Jul 23 15:27:01.021369 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.021348 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 23 15:27:01.039263 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.039237 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-lp2j9"] Jul 23 15:27:01.101393 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.101359 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-lhffw"] Jul 23 15:27:01.104744 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.104727 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-lhffw" Jul 23 15:27:01.110543 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.110518 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-g64gz\"" Jul 23 15:27:01.122287 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.122265 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-lhffw"] Jul 23 15:27:01.169358 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.169319 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch6xq\" (UniqueName: \"kubernetes.io/projected/91255298-1128-400d-9e9e-b5b560eac8aa-kube-api-access-ch6xq\") pod \"payload-pre-processing-7677b774bb-lp2j9\" (UID: \"91255298-1128-400d-9e9e-b5b560eac8aa\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-lp2j9" Jul 23 15:27:01.169517 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.169399 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91255298-1128-400d-9e9e-b5b560eac8aa-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-lp2j9\" (UID: \"91255298-1128-400d-9e9e-b5b560eac8aa\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-lp2j9" Jul 23 15:27:01.270857 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.270760 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch6xq\" (UniqueName: \"kubernetes.io/projected/91255298-1128-400d-9e9e-b5b560eac8aa-kube-api-access-ch6xq\") pod \"payload-pre-processing-7677b774bb-lp2j9\" (UID: \"91255298-1128-400d-9e9e-b5b560eac8aa\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-lp2j9" Jul 23 15:27:01.270857 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.270812 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e90e7269-815c-44fc-860c-25289abee306-plugins-config-volume\") pod \"payload-processing-5bf47cd988-lhffw\" (UID: \"e90e7269-815c-44fc-860c-25289abee306\") " pod="openshift-ingress/payload-processing-5bf47cd988-lhffw" Jul 23 15:27:01.270857 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.270843 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlv48\" (UniqueName: \"kubernetes.io/projected/e90e7269-815c-44fc-860c-25289abee306-kube-api-access-jlv48\") pod \"payload-processing-5bf47cd988-lhffw\" (UID: \"e90e7269-815c-44fc-860c-25289abee306\") " pod="openshift-ingress/payload-processing-5bf47cd988-lhffw" Jul 23 15:27:01.271141 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.270886 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91255298-1128-400d-9e9e-b5b560eac8aa-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-lp2j9\" (UID: \"91255298-1128-400d-9e9e-b5b560eac8aa\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-lp2j9" Jul 23 15:27:01.271712 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.271686 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91255298-1128-400d-9e9e-b5b560eac8aa-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-lp2j9\" (UID: \"91255298-1128-400d-9e9e-b5b560eac8aa\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-lp2j9" Jul 23 15:27:01.293289 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.293255 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch6xq\" (UniqueName: \"kubernetes.io/projected/91255298-1128-400d-9e9e-b5b560eac8aa-kube-api-access-ch6xq\") pod \"payload-pre-processing-7677b774bb-lp2j9\" (UID: \"91255298-1128-400d-9e9e-b5b560eac8aa\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-lp2j9" Jul 23 15:27:01.328400 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.328356 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-lp2j9" Jul 23 15:27:01.371672 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.371636 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e90e7269-815c-44fc-860c-25289abee306-plugins-config-volume\") pod \"payload-processing-5bf47cd988-lhffw\" (UID: \"e90e7269-815c-44fc-860c-25289abee306\") " pod="openshift-ingress/payload-processing-5bf47cd988-lhffw" Jul 23 15:27:01.371846 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.371687 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlv48\" (UniqueName: \"kubernetes.io/projected/e90e7269-815c-44fc-860c-25289abee306-kube-api-access-jlv48\") pod \"payload-processing-5bf47cd988-lhffw\" (UID: \"e90e7269-815c-44fc-860c-25289abee306\") " pod="openshift-ingress/payload-processing-5bf47cd988-lhffw" Jul 23 15:27:01.372478 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.372444 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e90e7269-815c-44fc-860c-25289abee306-plugins-config-volume\") pod \"payload-processing-5bf47cd988-lhffw\" (UID: \"e90e7269-815c-44fc-860c-25289abee306\") " pod="openshift-ingress/payload-processing-5bf47cd988-lhffw" Jul 23 15:27:01.393825 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.393764 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlv48\" (UniqueName: \"kubernetes.io/projected/e90e7269-815c-44fc-860c-25289abee306-kube-api-access-jlv48\") pod \"payload-processing-5bf47cd988-lhffw\" (UID: \"e90e7269-815c-44fc-860c-25289abee306\") " pod="openshift-ingress/payload-processing-5bf47cd988-lhffw" Jul 23 15:27:01.418534 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.418467 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-lhffw" Jul 23 15:27:01.505593 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.505568 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-lp2j9"] Jul 23 15:27:01.506739 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:27:01.506708 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91255298_1128_400d_9e9e_b5b560eac8aa.slice/crio-cb22d047387cb95e9ce875ecd0c2a9b8db7c0b98c73cc9548b79eea1f2af94e1 WatchSource:0}: Error finding container cb22d047387cb95e9ce875ecd0c2a9b8db7c0b98c73cc9548b79eea1f2af94e1: Status 404 returned error can't find the container with id cb22d047387cb95e9ce875ecd0c2a9b8db7c0b98c73cc9548b79eea1f2af94e1 Jul 23 15:27:01.642724 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:01.637906 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-lhffw"] Jul 23 15:27:02.309128 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:02.309084 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-lp2j9" event={"ID":"91255298-1128-400d-9e9e-b5b560eac8aa","Type":"ContainerStarted","Data":"cb22d047387cb95e9ce875ecd0c2a9b8db7c0b98c73cc9548b79eea1f2af94e1"} Jul 23 15:27:02.310971 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:02.310933 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-lhffw" event={"ID":"e90e7269-815c-44fc-860c-25289abee306","Type":"ContainerStarted","Data":"7d81bc5207811c12c9fa0cfd1e1c6ff74cf240f5ddfa86273b3f0b4b226d4b91"} Jul 23 15:27:05.330036 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:05.330004 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-lhffw" event={"ID":"e90e7269-815c-44fc-860c-25289abee306","Type":"ContainerStarted","Data":"795b24d99be3ffbeb7d55079cb8204afbefbd0d00aad7178e13591e78394bcd2"} Jul 23 15:27:05.330528 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:05.330082 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-lhffw" Jul 23 15:27:05.331706 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:05.331677 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-lp2j9" event={"ID":"91255298-1128-400d-9e9e-b5b560eac8aa","Type":"ContainerStarted","Data":"82f68ed451540e11d12f901e1220d4107a368127f9de06a470e995d52d2ad503"} Jul 23 15:27:05.331837 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:05.331797 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-lp2j9" Jul 23 15:27:05.351779 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:05.351737 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-lhffw" podStartSLOduration=1.202323694 podStartE2EDuration="4.351724622s" totalImagesPullingTime="3.149400928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:27:05.350335332 +0000 UTC m=+878.966382596" watchObservedRunningTime="2026-07-23 15:27:05.351724622 +0000 UTC m=+878.967771886" Jul 23 15:27:05.371403 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:05.371330 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-lp2j9" podStartSLOduration=2.098200226 podStartE2EDuration="5.371307008s" totalImagesPullingTime="3.273106782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:27:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:27:05.368522052 +0000 UTC m=+878.984569320" watchObservedRunningTime="2026-07-23 15:27:05.371307008 +0000 UTC m=+878.987354266" Jul 23 15:27:09.357009 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:09.356975 2570 generic.go:356] "Generic (PLEG): container finished" podID="bb897d59-bcb8-437a-8a31-706f28a1e6e6" containerID="7b2e87b17c822cfbf27025451456fc116f16090ceb991d49929dc26cbcdd9ed2" exitCode=0 Jul 23 15:27:09.357459 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:09.357007 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" event={"ID":"bb897d59-bcb8-437a-8a31-706f28a1e6e6","Type":"ContainerDied","Data":"7b2e87b17c822cfbf27025451456fc116f16090ceb991d49929dc26cbcdd9ed2"} Jul 23 15:27:09.357459 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:09.357413 2570 scope.go:117] "RemoveContainer" containerID="7b2e87b17c822cfbf27025451456fc116f16090ceb991d49929dc26cbcdd9ed2" Jul 23 15:27:09.358418 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:09.358387 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:27:09.358527 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:09.358456 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:27:09.358527 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:09.358504 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:27:10.367375 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:10.367339 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" event={"ID":"bb897d59-bcb8-437a-8a31-706f28a1e6e6","Type":"ContainerStarted","Data":"c1d43db99813894eb96ed1d949da1c5d181d1b1e7b74f76b7b8c64b4eaa3348e"} Jul 23 15:27:11.093689 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:11.093600 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:27:11.093859 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:11.093789 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-h5kf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 23 15:27:11.093859 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:11.093843 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" podUID="bb897d59-bcb8-437a-8a31-706f28a1e6e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 23 15:27:12.093931 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:12.093891 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-h5kf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 23 15:27:12.094267 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:12.093965 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" podUID="bb897d59-bcb8-437a-8a31-706f28a1e6e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 23 15:27:13.094420 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:13.094383 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-h5kf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 23 15:27:13.094839 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:13.094446 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" podUID="bb897d59-bcb8-437a-8a31-706f28a1e6e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 23 15:27:14.094406 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:14.094367 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-h5kf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 23 15:27:14.094591 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:14.094427 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" podUID="bb897d59-bcb8-437a-8a31-706f28a1e6e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 23 15:27:15.098115 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:15.098087 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:27:15.098628 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:15.098528 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:27:15.099099 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:15.099078 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-h5kf5" Jul 23 15:27:16.338796 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:16.338765 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-lhffw" Jul 23 15:27:16.339180 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:16.338838 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-lp2j9" Jul 23 15:27:18.651017 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:18.650914 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-f2l7w_9ede6ef5-5d6c-4523-a2c3-ab77e9d33277/manager/0.log" Jul 23 15:27:18.919109 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:18.919038 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-847795f4d4-8gz9z_95da00bc-38f0-4946-8a7a-abb805f693ec/manager/0.log" Jul 23 15:27:19.046793 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:19.046768 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7hg42_b820a833-0c40-47e7-a675-faf0f3eacd95/server/0.log" Jul 23 15:27:19.174096 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:19.174027 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kdk4x_a8b33059-f17d-4231-a25a-ef79efd07c39/manager/0.log" Jul 23 15:27:19.583356 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:19.583270 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-777bc59866-mbqvc_b532a34f-49c8-4b1f-b741-155d83d5215e/copy-manifests/0.log" Jul 23 15:27:19.591801 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:19.591780 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-777bc59866-mbqvc_b532a34f-49c8-4b1f-b741-155d83d5215e/manager/0.log" Jul 23 15:27:20.073051 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:20.073019 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-gtvcx_6d34fa9a-0b4f-409a-9f2a-d99f3c69042d/postgres/0.log" Jul 23 15:27:21.663868 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:21.663833 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-8w6hg_8828e7d3-2224-4a34-bbd9-1f5faf9dbdfa/manager/0.log" Jul 23 15:27:21.788239 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:21.788212 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-psr4q_42a32983-bbde-41bc-a3f1-efb28b298737/manager/0.log" Jul 23 15:27:21.907522 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:21.907496 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-zh7f4_78cbfd02-7da4-4ecd-8416-fe34927d9c63/kuadrant-console-plugin/0.log" Jul 23 15:27:22.033488 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:22.033420 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-djtvm_aa23ceb8-ae48-4455-be26-5fa54d650ebe/registry-server/0.log" Jul 23 15:27:22.444805 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:22.444773 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-bqc5l_263bc01f-e1e1-4329-9c95-2785b4818ede/manager/0.log" Jul 23 15:27:22.829733 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:22.829660 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4p6r6q_0b22cc29-2cb5-442c-a14f-84c0992bd6c9/istio-proxy/0.log" Jul 23 15:27:23.201817 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:23.201791 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77f87dcdfb-sg7b2_3ff19238-e3df-4b67-8819-167a11efeb1d/kube-auth-proxy/0.log" Jul 23 15:27:23.326157 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:23.326121 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-h5kf5_bb897d59-bcb8-437a-8a31-706f28a1e6e6/istio-proxy/1.log" Jul 23 15:27:23.449623 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:23.449592 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-lp2j9_91255298-1128-400d-9e9e-b5b560eac8aa/payload-pre-processing/0.log" Jul 23 15:27:23.569791 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:23.569720 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-lhffw_e90e7269-815c-44fc-860c-25289abee306/payload-processing/0.log" Jul 23 15:27:23.699692 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:23.699663 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74d5f4575d-4d249_f97ae7a3-b422-44a9-8942-752524470865/router/0.log" Jul 23 15:27:28.072857 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:28.072829 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s6czh_5522141f-9f92-48ad-a954-7c0bff719ebd/console-operator/1.log" Jul 23 15:27:28.076822 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:28.076797 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cdjsd_946dd7b0-9ff6-41fe-9515-3e1d5b2e3867/ovn-acl-logging/0.log" Jul 23 15:27:28.080250 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:28.080230 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s6czh_5522141f-9f92-48ad-a954-7c0bff719ebd/console-operator/1.log" Jul 23 15:27:28.083951 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:28.083937 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cdjsd_946dd7b0-9ff6-41fe-9515-3e1d5b2e3867/ovn-acl-logging/0.log" Jul 23 15:27:29.779942 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:29.779907 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9x4vs/must-gather-jrj8s"] Jul 23 15:27:29.783535 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:29.783518 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9x4vs/must-gather-jrj8s" Jul 23 15:27:29.786132 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:29.786110 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9x4vs\"/\"default-dockercfg-wrzgr\"" Jul 23 15:27:29.786236 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:29.786112 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9x4vs\"/\"openshift-service-ca.crt\"" Jul 23 15:27:29.787016 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:29.786998 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9x4vs\"/\"kube-root-ca.crt\"" Jul 23 15:27:29.802073 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:29.802044 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9x4vs/must-gather-jrj8s"] Jul 23 15:27:29.861885 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:29.861838 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7bb4a9af-0457-4766-a463-874c09779b57-must-gather-output\") pod \"must-gather-jrj8s\" (UID: \"7bb4a9af-0457-4766-a463-874c09779b57\") " pod="openshift-must-gather-9x4vs/must-gather-jrj8s" Jul 23 15:27:29.862073 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:29.861901 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-669tt\" (UniqueName: \"kubernetes.io/projected/7bb4a9af-0457-4766-a463-874c09779b57-kube-api-access-669tt\") pod \"must-gather-jrj8s\" (UID: \"7bb4a9af-0457-4766-a463-874c09779b57\") " pod="openshift-must-gather-9x4vs/must-gather-jrj8s" Jul 23 15:27:29.962838 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:29.962794 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7bb4a9af-0457-4766-a463-874c09779b57-must-gather-output\") pod \"must-gather-jrj8s\" (UID: \"7bb4a9af-0457-4766-a463-874c09779b57\") " pod="openshift-must-gather-9x4vs/must-gather-jrj8s" Jul 23 15:27:29.963025 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:29.962851 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-669tt\" (UniqueName: \"kubernetes.io/projected/7bb4a9af-0457-4766-a463-874c09779b57-kube-api-access-669tt\") pod \"must-gather-jrj8s\" (UID: \"7bb4a9af-0457-4766-a463-874c09779b57\") " pod="openshift-must-gather-9x4vs/must-gather-jrj8s" Jul 23 15:27:29.963143 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:29.963122 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7bb4a9af-0457-4766-a463-874c09779b57-must-gather-output\") pod \"must-gather-jrj8s\" (UID: \"7bb4a9af-0457-4766-a463-874c09779b57\") " pod="openshift-must-gather-9x4vs/must-gather-jrj8s" Jul 23 15:27:29.971669 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:29.971640 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-669tt\" (UniqueName: \"kubernetes.io/projected/7bb4a9af-0457-4766-a463-874c09779b57-kube-api-access-669tt\") pod \"must-gather-jrj8s\" (UID: \"7bb4a9af-0457-4766-a463-874c09779b57\") " pod="openshift-must-gather-9x4vs/must-gather-jrj8s" Jul 23 15:27:30.094706 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:30.094612 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9x4vs/must-gather-jrj8s" Jul 23 15:27:30.240326 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:30.240287 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9x4vs/must-gather-jrj8s"] Jul 23 15:27:30.242428 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:27:30.242398 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bb4a9af_0457_4766_a463_874c09779b57.slice/crio-9f594b98aa1b59c0de4924f44c2d5ef105012b8d6df8683874ea279fbaaf968f WatchSource:0}: Error finding container 9f594b98aa1b59c0de4924f44c2d5ef105012b8d6df8683874ea279fbaaf968f: Status 404 returned error can't find the container with id 9f594b98aa1b59c0de4924f44c2d5ef105012b8d6df8683874ea279fbaaf968f Jul 23 15:27:30.466867 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:30.466834 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9x4vs/must-gather-jrj8s" event={"ID":"7bb4a9af-0457-4766-a463-874c09779b57","Type":"ContainerStarted","Data":"9f594b98aa1b59c0de4924f44c2d5ef105012b8d6df8683874ea279fbaaf968f"} Jul 23 15:27:31.475268 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:31.475192 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9x4vs/must-gather-jrj8s" event={"ID":"7bb4a9af-0457-4766-a463-874c09779b57","Type":"ContainerStarted","Data":"f11555ae6358ac639736aa683e011e33f40581e1968eaa694115d9536e26339f"} Jul 23 15:27:31.475268 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:31.475238 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9x4vs/must-gather-jrj8s" event={"ID":"7bb4a9af-0457-4766-a463-874c09779b57","Type":"ContainerStarted","Data":"e0d0d1b1b0b28bc466ca79191a71397708cd0e842c1309953d9a33b604fc0809"} Jul 23 15:27:31.499876 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:31.499808 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9x4vs/must-gather-jrj8s" podStartSLOduration=1.519902939 podStartE2EDuration="2.499785427s" totalImagesPullingTime="979.882488ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:27:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:27:31.497134518 +0000 UTC m=+905.113181782" watchObservedRunningTime="2026-07-23 15:27:31.499785427 +0000 UTC m=+905.115832692" Jul 23 15:27:33.016816 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:33.016783 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wl9qs_7cff11e5-8f10-4219-9ed9-76a3c70061c8/global-pull-secret-syncer/0.log" Jul 23 15:27:33.149411 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:33.149385 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vnw4c_e73fdf83-f791-4a9b-ae02-4df8329352a2/konnectivity-agent/0.log" Jul 23 15:27:33.200381 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:33.200347 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-63.ec2.internal_b932a6350e4894a972718b80947507d2/haproxy/0.log" Jul 23 15:27:38.336922 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:38.336891 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-8w6hg_8828e7d3-2224-4a34-bbd9-1f5faf9dbdfa/manager/0.log" Jul 23 15:27:38.367285 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:38.367194 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-psr4q_42a32983-bbde-41bc-a3f1-efb28b298737/manager/0.log" Jul 23 15:27:38.405452 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:38.405366 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-zh7f4_78cbfd02-7da4-4ecd-8416-fe34927d9c63/kuadrant-console-plugin/0.log" Jul 23 15:27:38.435436 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:38.435323 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-djtvm_aa23ceb8-ae48-4455-be26-5fa54d650ebe/registry-server/0.log" Jul 23 15:27:38.594879 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:38.594480 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-bqc5l_263bc01f-e1e1-4329-9c95-2785b4818ede/manager/0.log" Jul 23 15:27:40.579158 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:40.579117 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-9mmv8_45875b2d-d207-49c6-8b86-aa50e0198929/cluster-monitoring-operator/0.log" Jul 23 15:27:40.909861 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:40.909835 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d5t78_19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa/node-exporter/0.log" Jul 23 15:27:40.941068 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:40.941022 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d5t78_19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa/kube-rbac-proxy/0.log" Jul 23 15:27:40.968262 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:40.968233 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d5t78_19d5a1c6-06e7-4f94-ae02-18f05b0bd9aa/init-textfile/0.log" Jul 23 15:27:42.834622 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:42.834595 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-9ljm6_aa8c3277-5337-4477-adf1-f2917985d495/networking-console-plugin/0.log" Jul 23 15:27:43.262814 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.262777 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf"] Jul 23 15:27:43.269635 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.269606 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.276737 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.276704 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf"] Jul 23 15:27:43.309881 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.309844 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8eb79953-9996-4379-ad27-30943909b98e-proc\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.310075 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.309889 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8eb79953-9996-4379-ad27-30943909b98e-podres\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.310075 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.309919 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8eb79953-9996-4379-ad27-30943909b98e-sys\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.310075 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.309941 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjb6c\" (UniqueName: \"kubernetes.io/projected/8eb79953-9996-4379-ad27-30943909b98e-kube-api-access-mjb6c\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.310212 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.310130 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8eb79953-9996-4379-ad27-30943909b98e-lib-modules\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.410833 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.410800 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8eb79953-9996-4379-ad27-30943909b98e-proc\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.410833 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.410841 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8eb79953-9996-4379-ad27-30943909b98e-podres\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.411089 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.410871 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8eb79953-9996-4379-ad27-30943909b98e-sys\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.411089 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.410894 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjb6c\" (UniqueName: \"kubernetes.io/projected/8eb79953-9996-4379-ad27-30943909b98e-kube-api-access-mjb6c\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.411089 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.410997 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8eb79953-9996-4379-ad27-30943909b98e-podres\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.411089 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.411009 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8eb79953-9996-4379-ad27-30943909b98e-sys\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.411089 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.411007 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8eb79953-9996-4379-ad27-30943909b98e-lib-modules\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.411365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.411106 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8eb79953-9996-4379-ad27-30943909b98e-lib-modules\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.411365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.411132 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8eb79953-9996-4379-ad27-30943909b98e-proc\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.429087 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.429061 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjb6c\" (UniqueName: \"kubernetes.io/projected/8eb79953-9996-4379-ad27-30943909b98e-kube-api-access-mjb6c\") pod \"perf-node-gather-daemonset-gz8hf\" (UID: \"8eb79953-9996-4379-ad27-30943909b98e\") " pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.514383 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.514267 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s6czh_5522141f-9f92-48ad-a954-7c0bff719ebd/console-operator/1.log" Jul 23 15:27:43.518552 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.518520 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-s6czh_5522141f-9f92-48ad-a954-7c0bff719ebd/console-operator/2.log" Jul 23 15:27:43.584726 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.584684 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:43.756772 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:43.756728 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf"] Jul 23 15:27:43.760441 ip-10-0-129-63 kubenswrapper[2570]: W0723 15:27:43.760407 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8eb79953_9996_4379_ad27_30943909b98e.slice/crio-9066eec88a7ffab2630cc9ade1453712d934d15df56bfb3389f30da0209c573c WatchSource:0}: Error finding container 9066eec88a7ffab2630cc9ade1453712d934d15df56bfb3389f30da0209c573c: Status 404 returned error can't find the container with id 9066eec88a7ffab2630cc9ade1453712d934d15df56bfb3389f30da0209c573c Jul 23 15:27:44.581557 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:44.581508 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" event={"ID":"8eb79953-9996-4379-ad27-30943909b98e","Type":"ContainerStarted","Data":"6b4c0d0a26c8cfc030b935ee304c4bde41627d64631ccc945f4cf9953033fc36"} Jul 23 15:27:44.582139 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:44.582120 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:44.582277 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:44.582261 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" event={"ID":"8eb79953-9996-4379-ad27-30943909b98e","Type":"ContainerStarted","Data":"9066eec88a7ffab2630cc9ade1453712d934d15df56bfb3389f30da0209c573c"} Jul 23 15:27:44.601220 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:44.601150 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" podStartSLOduration=1.601134383 podStartE2EDuration="1.601134383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:27:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:27:44.600157228 +0000 UTC m=+918.216204493" watchObservedRunningTime="2026-07-23 15:27:44.601134383 +0000 UTC m=+918.217181669" Jul 23 15:27:44.658597 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:44.658570 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-pnzkz_c59394de-e921-4c8e-ab5c-d310242a8149/volume-data-source-validator/0.log" Jul 23 15:27:45.901192 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:45.901165 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-whcfg_5e6d6e02-f435-4ccd-9b43-5c4f79d86b50/dns/0.log" Jul 23 15:27:45.932348 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:45.932320 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-whcfg_5e6d6e02-f435-4ccd-9b43-5c4f79d86b50/kube-rbac-proxy/0.log" Jul 23 15:27:46.013366 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:46.013336 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mwbjc_15c40f6c-249b-4f48-a8ae-f8b6da077e0e/dns-node-resolver/0.log" Jul 23 15:27:46.746949 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:46.746924 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k57jv_8322eaa3-685b-44b5-834f-ae04b6ec6566/node-ca/0.log" Jul 23 15:27:48.091706 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:48.091669 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4p6r6q_0b22cc29-2cb5-442c-a14f-84c0992bd6c9/istio-proxy/0.log" Jul 23 15:27:48.258523 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:48.258496 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77f87dcdfb-sg7b2_3ff19238-e3df-4b67-8819-167a11efeb1d/kube-auth-proxy/0.log" Jul 23 15:27:48.292814 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:48.292787 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-h5kf5_bb897d59-bcb8-437a-8a31-706f28a1e6e6/istio-proxy/1.log" Jul 23 15:27:48.293158 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:48.293136 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-h5kf5_bb897d59-bcb8-437a-8a31-706f28a1e6e6/istio-proxy/0.log" Jul 23 15:27:48.315686 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:48.315656 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-lp2j9_91255298-1128-400d-9e9e-b5b560eac8aa/payload-pre-processing/0.log" Jul 23 15:27:48.342359 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:48.342223 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-lhffw_e90e7269-815c-44fc-860c-25289abee306/payload-processing/0.log" Jul 23 15:27:48.384786 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:48.384755 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74d5f4575d-4d249_f97ae7a3-b422-44a9-8942-752524470865/router/0.log" Jul 23 15:27:49.129916 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:49.129871 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8sl9b_e8857fb3-2ba9-46ab-b37e-a0dc8f6f85d9/serve-healthcheck-canary/0.log" Jul 23 15:27:49.678710 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:49.678679 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-jhn9s_48561ae8-8b4f-4367-a2ef-20584d90b8e4/insights-operator/1.log" Jul 23 15:27:49.679124 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:49.679105 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-jhn9s_48561ae8-8b4f-4367-a2ef-20584d90b8e4/insights-operator/0.log" Jul 23 15:27:49.708841 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:49.708818 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fwzgb_9c43bfcd-1a27-4f40-a0c8-50e90817eb85/kube-rbac-proxy/0.log" Jul 23 15:27:49.731976 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:49.731951 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fwzgb_9c43bfcd-1a27-4f40-a0c8-50e90817eb85/exporter/0.log" Jul 23 15:27:49.783032 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:49.783002 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fwzgb_9c43bfcd-1a27-4f40-a0c8-50e90817eb85/extractor/0.log" Jul 23 15:27:51.607274 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:51.607241 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9x4vs/perf-node-gather-daemonset-gz8hf" Jul 23 15:27:52.303454 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:52.303424 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-f2l7w_9ede6ef5-5d6c-4523-a2c3-ab77e9d33277/manager/0.log" Jul 23 15:27:52.361333 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:52.361280 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-847795f4d4-8gz9z_95da00bc-38f0-4946-8a7a-abb805f693ec/manager/0.log" Jul 23 15:27:52.390936 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:52.390912 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7hg42_b820a833-0c40-47e7-a675-faf0f3eacd95/server/0.log" Jul 23 15:27:52.445893 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:52.445857 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kdk4x_a8b33059-f17d-4231-a25a-ef79efd07c39/manager/0.log" Jul 23 15:27:52.593626 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:52.593555 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-777bc59866-mbqvc_b532a34f-49c8-4b1f-b741-155d83d5215e/manager/0.log" Jul 23 15:27:52.618699 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:52.618674 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-777bc59866-mbqvc_b532a34f-49c8-4b1f-b741-155d83d5215e/copy-manifests/0.log" Jul 23 15:27:53.056402 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:53.056367 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4p6r6q_0b22cc29-2cb5-442c-a14f-84c0992bd6c9/istio-proxy/0.log" Jul 23 15:27:53.122045 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:53.122006 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77f87dcdfb-sg7b2_3ff19238-e3df-4b67-8819-167a11efeb1d/kube-auth-proxy/0.log" Jul 23 15:27:53.143154 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:53.143091 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-h5kf5_bb897d59-bcb8-437a-8a31-706f28a1e6e6/istio-proxy/1.log" Jul 23 15:27:53.143574 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:53.143553 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-h5kf5_bb897d59-bcb8-437a-8a31-706f28a1e6e6/istio-proxy/0.log" Jul 23 15:27:53.158521 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:53.158479 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-lp2j9_91255298-1128-400d-9e9e-b5b560eac8aa/payload-pre-processing/0.log" Jul 23 15:27:53.178310 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:53.178260 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-lhffw_e90e7269-815c-44fc-860c-25289abee306/payload-processing/0.log" Jul 23 15:27:53.206972 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:53.206940 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74d5f4575d-4d249_f97ae7a3-b422-44a9-8942-752524470865/router/0.log" Jul 23 15:27:54.450085 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:54.450054 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4p6r6q_0b22cc29-2cb5-442c-a14f-84c0992bd6c9/istio-proxy/0.log" Jul 23 15:27:54.521676 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:54.521612 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77f87dcdfb-sg7b2_3ff19238-e3df-4b67-8819-167a11efeb1d/kube-auth-proxy/0.log" Jul 23 15:27:54.545185 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:54.545117 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-h5kf5_bb897d59-bcb8-437a-8a31-706f28a1e6e6/istio-proxy/0.log" Jul 23 15:27:54.546647 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:54.546617 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-h5kf5_bb897d59-bcb8-437a-8a31-706f28a1e6e6/istio-proxy/1.log" Jul 23 15:27:54.564672 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:54.564604 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-lp2j9_91255298-1128-400d-9e9e-b5b560eac8aa/payload-pre-processing/0.log" Jul 23 15:27:54.587726 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:54.587700 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-lhffw_e90e7269-815c-44fc-860c-25289abee306/payload-processing/0.log" Jul 23 15:27:54.608880 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:54.608853 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74d5f4575d-4d249_f97ae7a3-b422-44a9-8942-752524470865/router/0.log" Jul 23 15:27:59.798382 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:59.798347 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-w5gjt_36dc10ed-f042-4994-8241-113f098485d6/kube-storage-version-migrator-operator/1.log" Jul 23 15:27:59.799151 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:27:59.799134 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-w5gjt_36dc10ed-f042-4994-8241-113f098485d6/kube-storage-version-migrator-operator/0.log" Jul 23 15:28:00.860365 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:28:00.860337 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4bhpc_8028e717-3208-45d4-a49f-117180037b85/kube-multus-additional-cni-plugins/0.log" Jul 23 15:28:00.886631 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:28:00.886605 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4bhpc_8028e717-3208-45d4-a49f-117180037b85/egress-router-binary-copy/0.log" Jul 23 15:28:00.909317 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:28:00.909269 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4bhpc_8028e717-3208-45d4-a49f-117180037b85/cni-plugins/0.log" Jul 23 15:28:00.931171 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:28:00.931145 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4bhpc_8028e717-3208-45d4-a49f-117180037b85/bond-cni-plugin/0.log" Jul 23 15:28:00.953507 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:28:00.953481 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4bhpc_8028e717-3208-45d4-a49f-117180037b85/routeoverride-cni/0.log" Jul 23 15:28:00.980478 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:28:00.980450 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4bhpc_8028e717-3208-45d4-a49f-117180037b85/whereabouts-cni-bincopy/0.log" Jul 23 15:28:01.009330 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:28:01.009277 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4bhpc_8028e717-3208-45d4-a49f-117180037b85/whereabouts-cni/0.log" Jul 23 15:28:01.440004 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:28:01.439974 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rdgrb_ddf831fb-25bd-459e-8632-8f44d11ecd8b/kube-multus/0.log" Jul 23 15:28:01.512732 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:28:01.512624 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-n5624_7b189e73-6f40-467e-9bf1-5505c5a7194c/network-metrics-daemon/0.log" Jul 23 15:28:01.536276 ip-10-0-129-63 kubenswrapper[2570]: I0723 15:28:01.536247 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-n5624_7b189e73-6f40-467e-9bf1-5505c5a7194c/kube-rbac-proxy/0.log"