Sep 10 17:37:55.131713 ip-10-0-130-84 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 10 17:37:55.131727 ip-10-0-130-84 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 10 17:37:55.131737 ip-10-0-130-84 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 10 17:37:55.131936 ip-10-0-130-84 systemd[1]: Failed to start Kubernetes Kubelet. Sep 10 17:38:05.249501 ip-10-0-130-84 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 10 17:38:05.249523 ip-10-0-130-84 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot b74d076238e3488ca8740ce9d4984527 -- Sep 10 17:39:55.814187 ip-10-0-130-84 systemd[1]: Starting Kubernetes Kubelet... Sep 10 17:39:56.241707 ip-10-0-130-84 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 17:39:56.241707 ip-10-0-130-84 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 17:39:56.241707 ip-10-0-130-84 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 17:39:56.241707 ip-10-0-130-84 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 17:39:56.241707 ip-10-0-130-84 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 17:39:56.242825 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.242747 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 17:39:56.250075 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250052 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 17:39:56.250075 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250070 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 17:39:56.250075 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250074 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 17:39:56.250075 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250078 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 17:39:56.250075 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250082 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250085 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250088 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250091 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250094 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250097 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250100 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250103 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250107 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250109 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250112 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250115 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250118 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250121 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250123 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250127 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250129 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250132 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250134 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250137 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 17:39:56.250277 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250140 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250142 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250145 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250148 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250150 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250153 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250155 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250159 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250164 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250167 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250170 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250172 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250175 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250177 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250181 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250184 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250188 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250192 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250195 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 17:39:56.250753 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250198 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250200 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250203 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250206 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250209 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250212 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250215 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250218 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250221 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250223 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250226 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250229 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250231 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250234 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250237 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250240 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250242 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250245 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250247 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250250 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 17:39:56.251231 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250252 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250255 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250258 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250263 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250266 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250270 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250272 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250277 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250281 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250284 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250287 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250290 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250292 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250295 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250298 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250300 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250303 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250305 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250308 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 17:39:56.251707 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250311 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250314 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250317 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250319 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250322 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250325 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250327 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250330 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250332 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250335 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250338 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250340 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250343 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250345 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250348 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250351 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.250353 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251079 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251086 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251089 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 17:39:56.252205 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251093 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251096 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251099 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251103 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251107 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251111 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251114 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251117 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251120 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251123 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251126 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251128 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251131 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251134 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251137 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251140 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251143 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251146 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251148 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 17:39:56.252679 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251151 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251153 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251156 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251159 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251162 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251164 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251167 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251170 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251172 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251175 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251177 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251180 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251183 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251186 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251189 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251191 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251194 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251197 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251199 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 17:39:56.253164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251202 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251204 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251207 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251209 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251212 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251215 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251217 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251220 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251222 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251226 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251228 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251231 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251233 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251236 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251239 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251241 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251244 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251247 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251249 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251252 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 17:39:56.253628 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251255 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251258 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251260 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251263 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251266 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251271 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251275 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251278 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251281 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251285 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251289 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251292 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251294 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251298 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251302 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251305 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251308 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251310 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251313 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 17:39:56.254119 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251315 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251318 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251321 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251324 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251327 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251331 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251333 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251336 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251338 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251341 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251344 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251347 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251350 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251352 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251355 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251357 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251360 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251365 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251368 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251754 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 17:39:56.254606 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251758 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251761 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251764 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251768 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251772 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251774 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251777 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251780 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251783 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251785 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251788 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251791 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251793 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251796 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251798 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251802 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251804 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251807 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251809 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251812 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 17:39:56.255128 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251814 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251817 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251820 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251823 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251826 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251828 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251831 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251833 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251836 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251839 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251842 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251845 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251848 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251850 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251853 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251855 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251858 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251860 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251863 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251866 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 17:39:56.255614 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251868 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251871 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251873 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251875 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251897 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251900 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251903 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251905 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251909 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251911 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251914 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251916 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251919 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251921 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251924 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251927 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251930 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251933 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251935 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 17:39:56.256125 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251938 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251941 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251944 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251947 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251950 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251953 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251955 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251958 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251962 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251965 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251969 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251972 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251974 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251977 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251980 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251982 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251985 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251987 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251990 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 17:39:56.256624 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251993 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251995 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.251998 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252001 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252003 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252006 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252008 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252011 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252014 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252017 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252019 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252024 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252027 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252031 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252034 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252037 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252040 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252043 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252046 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252049 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 17:39:56.257099 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252123 2565 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252157 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252163 2565 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252168 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252172 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252175 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252179 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252183 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252186 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252189 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252193 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252196 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252199 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252202 2565 flags.go:64] FLAG: --cgroup-root="" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252205 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252208 2565 flags.go:64] FLAG: --client-ca-file="" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252212 2565 flags.go:64] FLAG: --cloud-provider="external" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252215 2565 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252220 2565 flags.go:64] FLAG: --cluster-domain="" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252223 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252230 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252233 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252237 2565 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252240 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 17:39:56.257582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252243 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252246 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252249 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252252 2565 flags.go:64] FLAG: --contention-profiling="false" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252256 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252259 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252262 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252264 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252271 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252274 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252277 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252280 2565 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252283 2565 flags.go:64] FLAG: --enable-server="true" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252286 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252290 2565 flags.go:64] FLAG: --event-burst="100" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252293 2565 flags.go:64] FLAG: --event-qps="50" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252297 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252300 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252303 2565 flags.go:64] FLAG: --eviction-hard="" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252306 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252311 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252313 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252316 2565 flags.go:64] FLAG: --eviction-soft="" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252319 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252323 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 17:39:56.258188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252326 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252329 2565 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252332 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252336 2565 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252339 2565 flags.go:64] FLAG: --feature-gates="" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252343 2565 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252346 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252349 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252352 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252355 2565 flags.go:64] FLAG: --healthz-port="10248" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252358 2565 flags.go:64] FLAG: --help="false" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252361 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252364 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252367 2565 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252370 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252374 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252377 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252380 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252383 2565 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252386 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252388 2565 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252391 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252394 2565 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252397 2565 flags.go:64] FLAG: --kube-reserved="" Sep 10 17:39:56.258840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252400 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252402 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252406 2565 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252410 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252412 2565 flags.go:64] FLAG: --lock-file="" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252415 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252418 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252421 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252426 2565 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252430 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252432 2565 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252436 2565 flags.go:64] FLAG: --logging-format="text" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252439 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252443 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252446 2565 flags.go:64] FLAG: --manifest-url="" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252449 2565 flags.go:64] FLAG: --manifest-url-header="" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252453 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252456 2565 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252460 2565 flags.go:64] FLAG: --max-pods="110" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252463 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252466 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252469 2565 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252472 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252475 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252478 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 17:39:56.259529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252481 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252490 2565 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252493 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252496 2565 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252499 2565 flags.go:64] FLAG: --pod-cidr="" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252502 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252508 2565 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252511 2565 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252514 2565 flags.go:64] FLAG: --pods-per-core="0" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252517 2565 flags.go:64] FLAG: --port="10250" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252521 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252524 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c092f63065979dbb" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252527 2565 flags.go:64] FLAG: --qos-reserved="" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252530 2565 flags.go:64] FLAG: --read-only-port="10255" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252533 2565 flags.go:64] FLAG: --register-node="true" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252536 2565 flags.go:64] FLAG: --register-with-taints="" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252544 2565 flags.go:64] FLAG: --registry-burst="10" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252547 2565 flags.go:64] FLAG: --registry-qps="5" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252550 2565 flags.go:64] FLAG: --reserved-cpus="" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252554 2565 flags.go:64] FLAG: --reserved-memory="" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252558 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252561 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252564 2565 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252567 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252569 2565 flags.go:64] FLAG: --runonce="false" Sep 10 17:39:56.260168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252572 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252575 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252578 2565 flags.go:64] FLAG: --seccomp-default="false" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252581 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252584 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252587 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252590 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252593 2565 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252595 2565 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252598 2565 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252602 2565 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252605 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252608 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252610 2565 flags.go:64] FLAG: --system-cgroups="" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252613 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252618 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252621 2565 flags.go:64] FLAG: --tls-cert-file="" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252625 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252629 2565 flags.go:64] FLAG: --tls-min-version="" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252632 2565 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252635 2565 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252638 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252640 2565 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252643 2565 flags.go:64] FLAG: --v="2" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252647 2565 flags.go:64] FLAG: --version="false" Sep 10 17:39:56.260778 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252652 2565 flags.go:64] FLAG: --vmodule="" Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252657 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.252660 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252770 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252774 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252776 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252780 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252782 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252785 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252788 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252790 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252793 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252795 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252798 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252801 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252803 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252806 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252808 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252812 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 17:39:56.261440 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252816 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252819 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252822 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252824 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252827 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252831 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252834 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252837 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252839 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252842 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252844 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252847 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252849 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252852 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252856 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252859 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252861 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252864 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252868 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 17:39:56.261927 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252872 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252875 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252878 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252881 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252897 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252901 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252903 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252907 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252909 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252912 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252916 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252920 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252923 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252926 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252929 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252932 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252935 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252937 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252941 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252944 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 17:39:56.262400 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252946 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252949 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252951 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252954 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252956 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252959 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252962 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252965 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252968 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252971 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252974 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252976 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252979 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252981 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252984 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252987 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252989 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252992 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252994 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 17:39:56.262903 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252997 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.252999 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253002 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253004 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253007 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253009 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253012 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253014 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253017 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253020 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253022 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253026 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253029 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253031 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253034 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253036 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253039 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253042 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253044 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253047 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 17:39:56.263376 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253051 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253054 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253057 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253059 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.253062 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.253070 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.260175 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.260190 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.261116 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.261124 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261195 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261200 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261204 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261207 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261211 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 17:39:56.263864 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261214 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261216 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261219 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261222 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261224 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261227 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261230 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261233 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261235 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261238 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261242 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261246 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261249 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261252 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261255 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261258 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261261 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261264 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261267 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 17:39:56.264282 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261270 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261273 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261275 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261278 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261281 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261285 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261288 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261291 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261294 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261296 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261299 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261302 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261304 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261307 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261309 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261312 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261315 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261319 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261322 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261325 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 17:39:56.264751 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261328 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261331 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261334 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261337 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261340 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261344 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261347 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261349 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261352 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261355 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261357 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261360 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261363 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261365 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261368 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261371 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261374 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261377 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261383 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261386 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 17:39:56.265286 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261389 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261391 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261394 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261396 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261399 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261402 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261404 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261406 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261409 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261412 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261414 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261417 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261420 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261423 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261426 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261428 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261431 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261433 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261436 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261440 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 17:39:56.265773 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261443 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261446 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261449 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261452 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261456 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261459 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261461 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261464 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261466 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261469 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261471 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261474 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261477 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261480 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261483 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 17:39:56.266333 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.261488 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261610 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261615 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261619 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261624 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261627 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261630 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261632 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261635 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261638 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261641 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261644 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261647 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261649 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261652 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261654 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261657 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261659 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261662 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 17:39:56.266714 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261665 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261667 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261670 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261673 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261675 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261678 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261681 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261683 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261686 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261688 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261691 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261694 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261697 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261699 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261702 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261704 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261707 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261709 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261712 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 17:39:56.267194 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261715 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261717 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261721 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261724 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261727 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261729 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261732 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261735 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261737 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261740 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261743 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261745 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261748 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261751 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261753 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261756 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261758 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261761 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261764 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 17:39:56.267670 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261766 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261769 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261771 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261774 2565 feature_gate.go:328] unrecognized feature gate: Example Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261776 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261779 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261782 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261785 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261787 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261790 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261792 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261795 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261797 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261800 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261803 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261805 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261808 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261810 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261813 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261815 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261818 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 17:39:56.268198 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261820 2565 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261823 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261826 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261828 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261831 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261833 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261836 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261838 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261841 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261844 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261846 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261849 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261852 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261854 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261857 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261859 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261862 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261866 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261869 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 17:39:56.268691 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261872 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 17:39:56.269164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261875 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 17:39:56.269164 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:56.261877 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 17:39:56.269164 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.261895 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 17:39:56.269164 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.262000 2565 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 17:39:56.269164 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.266036 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 17:39:56.269164 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.267173 2565 server.go:1019] "Starting client certificate rotation" Sep 10 17:39:56.269164 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.267274 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 17:39:56.269164 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.267312 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 17:39:56.269793 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.269776 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 17:39:56.272096 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.272078 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 17:39:56.283227 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.283210 2565 log.go:25] "Validated CRI v1 runtime API" Sep 10 17:39:56.286141 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.286126 2565 log.go:25] "Validated CRI v1 image API" Sep 10 17:39:56.286979 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.286962 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 17:39:56.289824 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.289802 2565 fs.go:135] Filesystem UUIDs: map[79aee1c8-9406-445c-b921-1a8317bae7af:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a022a472-ac3d-4706-8e59-b2495c6b1a16:/dev/nvme0n1p4] Sep 10 17:39:56.289918 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.289822 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 17:39:56.295146 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.295041 2565 manager.go:216] Machine: {Timestamp:2026-09-10 17:39:56.293437845 +0000 UTC m=+0.357848567 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100070 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d9187278ff3318d59a105bca2f324 SystemUUID:ec2d9187-278f-f331-8d59-a105bca2f324 BootID:b74d0762-38e3-488c-a874-0ce9d4984527 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9b:5c:fa:49:f3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9b:5c:fa:49:f3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:a9:76:05:ff:56 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 17:39:56.295146 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.295144 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 17:39:56.295261 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.295249 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 17:39:56.298338 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.298313 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 17:39:56.298476 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.298340 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-84.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 17:39:56.299679 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.299667 2565 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 17:39:56.299722 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.299681 2565 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 17:39:56.299722 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.299690 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 17:39:56.300457 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.300447 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 17:39:56.300804 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.300789 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 17:39:56.301988 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.301976 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 17:39:56.305413 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.305401 2565 state_mem.go:36] "Initialized new in-memory state store" Sep 10 17:39:56.305513 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.305505 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 17:39:56.305549 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.305533 2565 kubelet.go:486] "Attempting to sync node with API server" Sep 10 17:39:56.305549 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.305543 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 17:39:56.305600 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.305557 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 17:39:56.305600 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.305567 2565 kubelet.go:398] "Adding apiserver pod source" Sep 10 17:39:56.305600 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.305575 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 17:39:56.305948 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.305936 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 17:39:56.305948 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.305955 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 17:39:56.308275 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.308260 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 17:39:56.310562 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.310550 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 17:39:56.310617 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.310579 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 17:39:56.311813 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.311801 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 17:39:56.311859 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.311820 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 17:39:56.311859 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.311826 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 17:39:56.311859 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.311831 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 17:39:56.311859 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.311844 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 17:39:56.311859 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.311850 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 17:39:56.312007 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.311863 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 17:39:56.312007 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.311868 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 17:39:56.312007 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.311874 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 17:39:56.312007 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.311899 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 17:39:56.312007 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.311917 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 17:39:56.312007 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.311923 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 17:39:56.312851 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.312838 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 17:39:56.312896 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.312855 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 17:39:56.315869 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.315847 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 17:39:56.315869 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.315847 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-84.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 17:39:56.316837 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.316825 2565 server.go:1268] "Started kubelet" Sep 10 17:39:56.316941 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.316914 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 17:39:56.317044 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.316912 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 17:39:56.317044 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.316976 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 17:39:56.321483 ip-10-0-130-84 systemd[1]: Started Kubernetes Kubelet. Sep 10 17:39:56.321815 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.321571 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 17:39:56.323534 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.323421 2565 server.go:310] "Adding debug handlers to kubelet server" Sep 10 17:39:56.324022 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.323996 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 17:39:56.326813 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.326794 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 17:39:56.327236 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.327218 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 17:39:56.327350 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.327327 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-84.ec2.internal\" not found" Sep 10 17:39:56.328014 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.327994 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 17:39:56.328932 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.328914 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 17:39:56.328932 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.328932 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 17:39:56.329088 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.329038 2565 reconstruct.go:97] "Volume reconstruction finished" Sep 10 17:39:56.329088 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.329050 2565 reconciler.go:29] "Reconciler: start to sync state" Sep 10 17:39:56.329425 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.329397 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-84.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 17:39:56.329536 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.329515 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 17:39:56.330125 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.329140 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-84.ec2.internal.18d40668239d2fbe default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-84.ec2.internal,UID:ip-10-0-130-84.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-84.ec2.internal,},FirstTimestamp:2026-09-10 17:39:56.31679891 +0000 UTC m=+0.381209624,LastTimestamp:2026-09-10 17:39:56.31679891 +0000 UTC m=+0.381209624,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-84.ec2.internal,}" Sep 10 17:39:56.330198 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.330172 2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-84.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 17:39:56.330915 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.330899 2565 factory.go:153] Registering CRI-O factory Sep 10 17:39:56.330985 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.330920 2565 factory.go:223] Registration of the crio container factory successfully Sep 10 17:39:56.330985 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.330976 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 17:39:56.330985 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.330985 2565 factory.go:55] Registering systemd factory Sep 10 17:39:56.331085 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.330992 2565 factory.go:223] Registration of the systemd container factory successfully Sep 10 17:39:56.331085 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.331007 2565 factory.go:103] Registering Raw factory Sep 10 17:39:56.331085 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.331019 2565 manager.go:1196] Started watching for new ooms in manager Sep 10 17:39:56.331361 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.331351 2565 manager.go:349] Starting recovery of all containers Sep 10 17:39:56.335243 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.335216 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 17:39:56.341343 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.340551 2565 manager.go:354] Recovery completed Sep 10 17:39:56.345537 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.345523 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 17:39:56.347361 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.347343 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mkhx4" Sep 10 17:39:56.347493 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.347474 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasSufficientMemory" Sep 10 17:39:56.347581 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.347506 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 17:39:56.347581 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.347520 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasSufficientPID" Sep 10 17:39:56.348075 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.348036 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 17:39:56.348075 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.348047 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 17:39:56.348075 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.348065 2565 state_mem.go:36] "Initialized new in-memory state store" Sep 10 17:39:56.348190 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.348095 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 17:39:56.348766 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.348752 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasSufficientMemory" Sep 10 17:39:56.348823 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.348779 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 17:39:56.348823 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.348789 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasSufficientPID" Sep 10 17:39:56.349098 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.349021 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-84.ec2.internal.18d4066825718203 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-84.ec2.internal,UID:ip-10-0-130-84.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-84.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-84.ec2.internal,},FirstTimestamp:2026-09-10 17:39:56.347490819 +0000 UTC m=+0.411901531,LastTimestamp:2026-09-10 17:39:56.347490819 +0000 UTC m=+0.411901531,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-84.ec2.internal,}" Sep 10 17:39:56.350070 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.350055 2565 policy_none.go:49] "None policy: Start" Sep 10 17:39:56.350127 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.350077 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 17:39:56.350127 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.350091 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 17:39:56.352095 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.352080 2565 policy_none.go:47] "Start" Sep 10 17:39:56.359576 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.359552 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mkhx4" Sep 10 17:39:56.362303 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.361400 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-84.ec2.internal.18d406682571e088 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-84.ec2.internal,UID:ip-10-0-130-84.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-84.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-84.ec2.internal,},FirstTimestamp:2026-09-10 17:39:56.347515016 +0000 UTC m=+0.411925720,LastTimestamp:2026-09-10 17:39:56.347515016 +0000 UTC m=+0.411925720,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-84.ec2.internal,}" Sep 10 17:39:56.375733 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.375716 2565 manager.go:337] "Starting Device Plugin manager" Sep 10 17:39:56.375800 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.375752 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 17:39:56.375800 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.375766 2565 server.go:87] "Starting device plugin registration server" Sep 10 17:39:56.376046 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.376033 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 17:39:56.376096 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.376049 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 17:39:56.376236 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.376221 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 17:39:56.376310 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.376293 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 17:39:56.376310 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.376304 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 17:39:56.376724 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.376704 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 17:39:56.376798 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.376760 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-84.ec2.internal\" not found" Sep 10 17:39:56.430998 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.430972 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 17:39:56.431135 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.431027 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 17:39:56.431135 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.431053 2565 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 17:39:56.431135 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.431094 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 17:39:56.433571 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.433544 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 17:39:56.476654 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.476633 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 17:39:56.478484 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.478465 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasSufficientMemory" Sep 10 17:39:56.478574 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.478502 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 17:39:56.478574 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.478520 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasSufficientPID" Sep 10 17:39:56.478574 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.478548 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.499298 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.499230 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.499298 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.499254 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-84.ec2.internal\": node \"ip-10-0-130-84.ec2.internal\" not found" Sep 10 17:39:56.532117 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.532091 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-84.ec2.internal"] Sep 10 17:39:56.532223 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.532162 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 17:39:56.533075 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.533052 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasSufficientMemory" Sep 10 17:39:56.533075 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.533078 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 17:39:56.533194 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.533092 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasSufficientPID" Sep 10 17:39:56.535277 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.535266 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 17:39:56.535397 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.535386 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.535430 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.535415 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 17:39:56.535992 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.535972 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasSufficientMemory" Sep 10 17:39:56.535992 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.535992 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 17:39:56.536095 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.536002 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasSufficientPID" Sep 10 17:39:56.536095 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.536034 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasSufficientMemory" Sep 10 17:39:56.536095 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.536055 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 17:39:56.536095 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.536066 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasSufficientPID" Sep 10 17:39:56.538048 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.538032 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.538114 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.538058 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 17:39:56.538677 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.538662 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasSufficientMemory" Sep 10 17:39:56.538761 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.538691 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 17:39:56.538761 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.538707 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeHasSufficientPID" Sep 10 17:39:56.547460 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.547443 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-84.ec2.internal\" not found" Sep 10 17:39:56.564502 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.564478 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-84.ec2.internal\" not found" node="ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.568701 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.568686 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-84.ec2.internal\" not found" node="ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.630555 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.630528 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3c069671a249c5c5f8caa70b379bea25-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal\" (UID: \"3c069671a249c5c5f8caa70b379bea25\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.630555 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.630557 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c069671a249c5c5f8caa70b379bea25-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal\" (UID: \"3c069671a249c5c5f8caa70b379bea25\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.630704 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.630574 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b0f5917d8bc06c28727df72b1ab49ae9-config\") pod \"kube-apiserver-proxy-ip-10-0-130-84.ec2.internal\" (UID: \"b0f5917d8bc06c28727df72b1ab49ae9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.648353 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.648331 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-84.ec2.internal\" not found" Sep 10 17:39:56.731697 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.731669 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3c069671a249c5c5f8caa70b379bea25-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal\" (UID: \"3c069671a249c5c5f8caa70b379bea25\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.731837 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.731716 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c069671a249c5c5f8caa70b379bea25-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal\" (UID: \"3c069671a249c5c5f8caa70b379bea25\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.731837 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.731727 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3c069671a249c5c5f8caa70b379bea25-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal\" (UID: \"3c069671a249c5c5f8caa70b379bea25\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.731837 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.731744 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b0f5917d8bc06c28727df72b1ab49ae9-config\") pod \"kube-apiserver-proxy-ip-10-0-130-84.ec2.internal\" (UID: \"b0f5917d8bc06c28727df72b1ab49ae9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.731837 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.731785 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b0f5917d8bc06c28727df72b1ab49ae9-config\") pod \"kube-apiserver-proxy-ip-10-0-130-84.ec2.internal\" (UID: \"b0f5917d8bc06c28727df72b1ab49ae9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.731837 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.731796 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c069671a249c5c5f8caa70b379bea25-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal\" (UID: \"3c069671a249c5c5f8caa70b379bea25\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.748539 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.748521 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-84.ec2.internal\" not found" Sep 10 17:39:56.849596 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.849532 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-84.ec2.internal\" not found" Sep 10 17:39:56.867683 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.867664 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.870914 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:56.870868 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-84.ec2.internal" Sep 10 17:39:56.950675 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:56.950645 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-84.ec2.internal\" not found" Sep 10 17:39:57.051175 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:57.051153 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-84.ec2.internal\" not found" Sep 10 17:39:57.151699 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:57.151463 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-84.ec2.internal\" not found" Sep 10 17:39:57.218722 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.218694 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 17:39:57.228309 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.228288 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal" Sep 10 17:39:57.242597 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.242573 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 17:39:57.245063 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.245043 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-84.ec2.internal" Sep 10 17:39:57.256099 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.256078 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 17:39:57.267322 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.267299 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 17:39:57.267476 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.267456 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 17:39:57.267532 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.267474 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 17:39:57.305980 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.305945 2565 apiserver.go:52] "Watching apiserver" Sep 10 17:39:57.324995 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.324962 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 17:39:57.326210 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.326187 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-s42t8","kube-system/konnectivity-agent-2tz6j","kube-system/kube-apiserver-proxy-ip-10-0-130-84.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r","openshift-dns/node-resolver-w2k5w","openshift-image-registry/node-ca-66qw8","openshift-multus/multus-additional-cni-plugins-r4tz2","openshift-multus/network-metrics-daemon-cvx9n","openshift-cluster-node-tuning-operator/tuned-vgrm9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal","openshift-multus/multus-nj8jg","openshift-network-diagnostics/network-check-target-5k8rp","openshift-network-operator/iptables-alerter-xnch8","openshift-ovn-kubernetes/ovnkube-node-g2pbs"] Sep 10 17:39:57.327132 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.327117 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 17:39:57.329540 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.329448 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.331715 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.331297 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xnch8" Sep 10 17:39:57.332512 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.332472 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 17:39:57.332635 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.332581 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 17:39:57.333106 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.333084 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 17:39:57.333215 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.333140 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 17:39:57.333390 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.333360 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xdph4\"" Sep 10 17:39:57.333502 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.333405 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 17:39:57.334541 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.334516 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 17:39:57.335290 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.335258 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3708abb-21c9-4590-90a9-00cfec61f382-system-cni-dir\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.335395 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.335294 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3708abb-21c9-4590-90a9-00cfec61f382-cnibin\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.335395 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.335322 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3708abb-21c9-4590-90a9-00cfec61f382-cni-binary-copy\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.335395 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.335345 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3708abb-21c9-4590-90a9-00cfec61f382-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.335395 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.335370 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e3708abb-21c9-4590-90a9-00cfec61f382-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.335633 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.335426 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5df73191-ea82-4144-83bf-57c0d770a4a2-host-slash\") pod \"iptables-alerter-xnch8\" (UID: \"5df73191-ea82-4144-83bf-57c0d770a4a2\") " pod="openshift-network-operator/iptables-alerter-xnch8" Sep 10 17:39:57.335633 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.335474 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e3708abb-21c9-4590-90a9-00cfec61f382-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.335633 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.335502 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnjtt\" (UniqueName: \"kubernetes.io/projected/e3708abb-21c9-4590-90a9-00cfec61f382-kube-api-access-jnjtt\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.335633 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.335527 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5df73191-ea82-4144-83bf-57c0d770a4a2-iptables-alerter-script\") pod \"iptables-alerter-xnch8\" (UID: \"5df73191-ea82-4144-83bf-57c0d770a4a2\") " pod="openshift-network-operator/iptables-alerter-xnch8" Sep 10 17:39:57.335633 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.335550 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2s64\" (UniqueName: \"kubernetes.io/projected/5df73191-ea82-4144-83bf-57c0d770a4a2-kube-api-access-q2s64\") pod \"iptables-alerter-xnch8\" (UID: \"5df73191-ea82-4144-83bf-57c0d770a4a2\") " pod="openshift-network-operator/iptables-alerter-xnch8" Sep 10 17:39:57.335882 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.335648 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.336573 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.336556 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 17:39:57.336731 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.336712 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8kh2s\"" Sep 10 17:39:57.336824 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.336808 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 17:39:57.337817 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.337798 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.338871 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.338848 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 17:39:57.339021 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.338880 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 17:39:57.339197 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.339180 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 17:39:57.340144 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.340119 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 17:39:57.340144 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.340133 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 17:39:57.340333 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.340215 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 17:39:57.340456 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.340440 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vh96r\"" Sep 10 17:39:57.340523 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.340494 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 17:39:57.340584 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.340518 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w2k5w" Sep 10 17:39:57.340584 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.340566 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 17:39:57.340705 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.340567 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w5v8n\"" Sep 10 17:39:57.341054 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.340957 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 17:39:57.342766 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.342748 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 17:39:57.343103 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.343080 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-66qw8" Sep 10 17:39:57.343185 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.343133 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 17:39:57.343315 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.343291 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 17:39:57.344079 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.344061 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9r6zz\"" Sep 10 17:39:57.345322 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.345304 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.347483 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.347461 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:39:57.347563 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:57.347546 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:39:57.348487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.348471 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 17:39:57.348627 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.348611 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rv5zr\"" Sep 10 17:39:57.349585 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.349558 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 17:39:57.349686 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.349671 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 17:39:57.349929 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.349897 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2tz6j" Sep 10 17:39:57.350100 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.350076 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 17:39:57.350179 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.350085 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j59cz\"" Sep 10 17:39:57.350422 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.350391 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 17:39:57.352937 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.352919 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kft6h\"" Sep 10 17:39:57.353039 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.352998 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:39:57.353097 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:57.353062 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:39:57.353214 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.353200 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 17:39:57.353274 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.353224 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 17:39:57.355505 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.355488 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.357370 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.357353 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 17:39:57.357590 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.357576 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4zdl9\"" Sep 10 17:39:57.358055 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.358041 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:39:57.358116 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:57.358099 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:39:57.363554 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.363527 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 17:34:57 +0000 UTC" deadline="2028-02-07 03:15:56.480606156 +0000 UTC" Sep 10 17:39:57.363624 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.363568 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12345h35m59.117057394s" Sep 10 17:39:57.369669 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.369647 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9rv2s" Sep 10 17:39:57.378291 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.378269 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9rv2s" Sep 10 17:39:57.429010 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.428986 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 17:39:57.435917 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.435898 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jctb\" (UniqueName: \"kubernetes.io/projected/b563ccdb-296e-4358-81f7-82dc6bd66435-kube-api-access-4jctb\") pod \"network-metrics-daemon-cvx9n\" (UID: \"b563ccdb-296e-4358-81f7-82dc6bd66435\") " pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:39:57.436026 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.435927 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggcw7\" (UniqueName: \"kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7\") pod \"network-check-target-5k8rp\" (UID: \"c3d44764-8e31-4aaf-a532-86de76fc6820\") " pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:39:57.436026 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.435948 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-slash\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.436026 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.435995 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-system-cni-dir\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.436026 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436012 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/588747e3-bc35-4a33-9545-3c606ef41faa-cni-binary-copy\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.436026 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436026 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dc0c8edc-89e0-4448-896c-78aa56a57d1c-serviceca\") pod \"node-ca-66qw8\" (UID: \"dc0c8edc-89e0-4448-896c-78aa56a57d1c\") " pod="openshift-image-registry/node-ca-66qw8" Sep 10 17:39:57.436282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436050 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvbsg\" (UniqueName: \"kubernetes.io/projected/dc0c8edc-89e0-4448-896c-78aa56a57d1c-kube-api-access-kvbsg\") pod \"node-ca-66qw8\" (UID: \"dc0c8edc-89e0-4448-896c-78aa56a57d1c\") " pod="openshift-image-registry/node-ca-66qw8" Sep 10 17:39:57.436282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436096 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-lib-modules\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.436282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436124 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-var-lib-kubelet\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.436282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436151 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-sysctl-conf\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.436282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436169 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/af672f7a-0702-4a9b-8a25-5cab8441f3c2-kubelet-config\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:39:57.436282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436189 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-run-systemd\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.436282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436208 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-cni-netd\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.436282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436241 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/588747e3-bc35-4a33-9545-3c606ef41faa-multus-daemon-config\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.436282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436269 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-socket-dir\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.436282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436286 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-tuned\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.436707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436338 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8wzk\" (UniqueName: \"kubernetes.io/projected/88785664-b748-45f4-a3bd-c6baffa832b0-kube-api-access-p8wzk\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.436707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436365 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-sysconfig\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.436707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436381 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-sys\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.436707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436401 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-host\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.436707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436423 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mltlv\" (UniqueName: \"kubernetes.io/projected/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-kube-api-access-mltlv\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.436707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436459 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/af672f7a-0702-4a9b-8a25-5cab8441f3c2-dbus\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:39:57.436707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436478 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-systemd-units\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.436707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436500 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-etc-kubernetes\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.436707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436523 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-var-lib-openvswitch\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.436707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436549 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-etc-openvswitch\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.436707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436575 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-node-log\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.436707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436598 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-multus-cni-dir\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.436707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436621 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-multus-socket-dir-parent\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.436707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436650 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-device-dir\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436711 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-run-netns\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436753 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnjtt\" (UniqueName: \"kubernetes.io/projected/e3708abb-21c9-4590-90a9-00cfec61f382-kube-api-access-jnjtt\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436777 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-run-k8s-cni-cncf-io\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436799 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-tmp\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436817 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f309e28c-ca0c-40e2-b6b7-3beb2f3213c7-konnectivity-ca\") pod \"konnectivity-agent-2tz6j\" (UID: \"f309e28c-ca0c-40e2-b6b7-3beb2f3213c7\") " pod="kube-system/konnectivity-agent-2tz6j" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436852 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f558fde8-d070-4538-bb97-19bb59832b85-ovnkube-script-lib\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436910 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3708abb-21c9-4590-90a9-00cfec61f382-cni-binary-copy\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.436937 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-var-lib-cni-multus\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437026 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5df73191-ea82-4144-83bf-57c0d770a4a2-iptables-alerter-script\") pod \"iptables-alerter-xnch8\" (UID: \"5df73191-ea82-4144-83bf-57c0d770a4a2\") " pod="openshift-network-operator/iptables-alerter-xnch8" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437057 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-run-netns\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437101 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-run-multus-certs\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437128 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-run\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437193 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e3708abb-21c9-4590-90a9-00cfec61f382-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5df73191-ea82-4144-83bf-57c0d770a4a2-host-slash\") pod \"iptables-alerter-xnch8\" (UID: \"5df73191-ea82-4144-83bf-57c0d770a4a2\") " pod="openshift-network-operator/iptables-alerter-xnch8" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437292 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh9qd\" (UniqueName: \"kubernetes.io/projected/0c6a2e31-247f-4534-845e-8e68feecb206-kube-api-access-dh9qd\") pod \"node-resolver-w2k5w\" (UID: \"0c6a2e31-247f-4534-845e-8e68feecb206\") " pod="openshift-dns/node-resolver-w2k5w" Sep 10 17:39:57.437364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437317 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-cnibin\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437341 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-registration-dir\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437375 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-kubelet\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437400 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-run-ovn-kubernetes\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437459 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f558fde8-d070-4538-bb97-19bb59832b85-ovn-node-metrics-cert\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437486 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-var-lib-cni-bin\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437510 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2r7f\" (UniqueName: \"kubernetes.io/projected/588747e3-bc35-4a33-9545-3c606ef41faa-kube-api-access-t2r7f\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437533 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc0c8edc-89e0-4448-896c-78aa56a57d1c-host\") pod \"node-ca-66qw8\" (UID: \"dc0c8edc-89e0-4448-896c-78aa56a57d1c\") " pod="openshift-image-registry/node-ca-66qw8" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437558 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-systemd\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437584 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437588 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3708abb-21c9-4590-90a9-00cfec61f382-cni-binary-copy\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437647 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5df73191-ea82-4144-83bf-57c0d770a4a2-host-slash\") pod \"iptables-alerter-xnch8\" (UID: \"5df73191-ea82-4144-83bf-57c0d770a4a2\") " pod="openshift-network-operator/iptables-alerter-xnch8" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437654 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-cni-bin\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437685 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5df73191-ea82-4144-83bf-57c0d770a4a2-iptables-alerter-script\") pod \"iptables-alerter-xnch8\" (UID: \"5df73191-ea82-4144-83bf-57c0d770a4a2\") " pod="openshift-network-operator/iptables-alerter-xnch8" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437688 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3708abb-21c9-4590-90a9-00cfec61f382-system-cni-dir\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437726 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3708abb-21c9-4590-90a9-00cfec61f382-system-cni-dir\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.438187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437731 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3708abb-21c9-4590-90a9-00cfec61f382-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437770 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-var-lib-kubelet\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437798 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2s64\" (UniqueName: \"kubernetes.io/projected/5df73191-ea82-4144-83bf-57c0d770a4a2-kube-api-access-q2s64\") pod \"iptables-alerter-xnch8\" (UID: \"5df73191-ea82-4144-83bf-57c0d770a4a2\") " pod="openshift-network-operator/iptables-alerter-xnch8" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437826 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0c6a2e31-247f-4534-845e-8e68feecb206-hosts-file\") pod \"node-resolver-w2k5w\" (UID: \"0c6a2e31-247f-4534-845e-8e68feecb206\") " pod="openshift-dns/node-resolver-w2k5w" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437857 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-hostroot\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437864 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3708abb-21c9-4590-90a9-00cfec61f382-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437884 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437909 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-sysctl-d\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.437973 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-run-ovn\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438001 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f558fde8-d070-4538-bb97-19bb59832b85-ovnkube-config\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438027 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crgj7\" (UniqueName: \"kubernetes.io/projected/f558fde8-d070-4538-bb97-19bb59832b85-kube-api-access-crgj7\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438054 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3708abb-21c9-4590-90a9-00cfec61f382-cnibin\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438091 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs\") pod \"network-metrics-daemon-cvx9n\" (UID: \"b563ccdb-296e-4358-81f7-82dc6bd66435\") " pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438101 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3708abb-21c9-4590-90a9-00cfec61f382-cnibin\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438118 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-multus-conf-dir\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438125 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e3708abb-21c9-4590-90a9-00cfec61f382-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.438798 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438151 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.439351 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438173 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-sys-fs\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.439351 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438188 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-log-socket\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.439351 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438208 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.439351 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438224 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f558fde8-d070-4538-bb97-19bb59832b85-env-overrides\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.439351 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438252 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e3708abb-21c9-4590-90a9-00cfec61f382-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.439351 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438275 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c6a2e31-247f-4534-845e-8e68feecb206-tmp-dir\") pod \"node-resolver-w2k5w\" (UID: \"0c6a2e31-247f-4534-845e-8e68feecb206\") " pod="openshift-dns/node-resolver-w2k5w" Sep 10 17:39:57.439351 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438290 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-modprobe-d\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.439351 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438305 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-kubernetes\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.439351 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438321 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f309e28c-ca0c-40e2-b6b7-3beb2f3213c7-agent-certs\") pod \"konnectivity-agent-2tz6j\" (UID: \"f309e28c-ca0c-40e2-b6b7-3beb2f3213c7\") " pod="kube-system/konnectivity-agent-2tz6j" Sep 10 17:39:57.439351 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438343 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-run-openvswitch\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.439351 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.438664 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e3708abb-21c9-4590-90a9-00cfec61f382-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.449992 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.449948 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 17:39:57.453802 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.453778 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnjtt\" (UniqueName: \"kubernetes.io/projected/e3708abb-21c9-4590-90a9-00cfec61f382-kube-api-access-jnjtt\") pod \"multus-additional-cni-plugins-r4tz2\" (UID: \"e3708abb-21c9-4590-90a9-00cfec61f382\") " pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.453948 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.453776 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2s64\" (UniqueName: \"kubernetes.io/projected/5df73191-ea82-4144-83bf-57c0d770a4a2-kube-api-access-q2s64\") pod \"iptables-alerter-xnch8\" (UID: \"5df73191-ea82-4144-83bf-57c0d770a4a2\") " pod="openshift-network-operator/iptables-alerter-xnch8" Sep 10 17:39:57.456379 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:57.456333 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c069671a249c5c5f8caa70b379bea25.slice/crio-cc739e937766c20c00f46b455367a2399261683eb1434a5b509be3806e1d4b60 WatchSource:0}: Error finding container cc739e937766c20c00f46b455367a2399261683eb1434a5b509be3806e1d4b60: Status 404 returned error can't find the container with id cc739e937766c20c00f46b455367a2399261683eb1434a5b509be3806e1d4b60 Sep 10 17:39:57.457053 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:57.457034 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f5917d8bc06c28727df72b1ab49ae9.slice/crio-2bf580594af14cd7e61adbbe323aaed54a59703b70ca44e61cdff8c422f25706 WatchSource:0}: Error finding container 2bf580594af14cd7e61adbbe323aaed54a59703b70ca44e61cdff8c422f25706: Status 404 returned error can't find the container with id 2bf580594af14cd7e61adbbe323aaed54a59703b70ca44e61cdff8c422f25706 Sep 10 17:39:57.462844 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.462827 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 17:39:57.538786 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.538752 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-system-cni-dir\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.538786 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.538790 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/588747e3-bc35-4a33-9545-3c606ef41faa-cni-binary-copy\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.539032 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.538805 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dc0c8edc-89e0-4448-896c-78aa56a57d1c-serviceca\") pod \"node-ca-66qw8\" (UID: \"dc0c8edc-89e0-4448-896c-78aa56a57d1c\") " pod="openshift-image-registry/node-ca-66qw8" Sep 10 17:39:57.539032 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.538821 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvbsg\" (UniqueName: \"kubernetes.io/projected/dc0c8edc-89e0-4448-896c-78aa56a57d1c-kube-api-access-kvbsg\") pod \"node-ca-66qw8\" (UID: \"dc0c8edc-89e0-4448-896c-78aa56a57d1c\") " pod="openshift-image-registry/node-ca-66qw8" Sep 10 17:39:57.539032 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.538961 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-system-cni-dir\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.539032 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.538966 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-lib-modules\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.539032 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539014 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-var-lib-kubelet\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539044 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-sysctl-conf\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539068 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/af672f7a-0702-4a9b-8a25-5cab8441f3c2-kubelet-config\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539084 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-run-systemd\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539107 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-cni-netd\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539139 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-lib-modules\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539147 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/af672f7a-0702-4a9b-8a25-5cab8441f3c2-kubelet-config\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539162 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-run-systemd\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539163 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-var-lib-kubelet\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539184 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/588747e3-bc35-4a33-9545-3c606ef41faa-multus-daemon-config\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539207 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-sysctl-conf\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539216 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-socket-dir\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539212 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-cni-netd\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539255 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-tuned\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539277 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8wzk\" (UniqueName: \"kubernetes.io/projected/88785664-b748-45f4-a3bd-c6baffa832b0-kube-api-access-p8wzk\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.539301 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539294 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-sysconfig\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539318 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-sys\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539329 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dc0c8edc-89e0-4448-896c-78aa56a57d1c-serviceca\") pod \"node-ca-66qw8\" (UID: \"dc0c8edc-89e0-4448-896c-78aa56a57d1c\") " pod="openshift-image-registry/node-ca-66qw8" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539341 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-host\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539365 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mltlv\" (UniqueName: \"kubernetes.io/projected/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-kube-api-access-mltlv\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539380 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-socket-dir\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539389 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/af672f7a-0702-4a9b-8a25-5cab8441f3c2-dbus\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539399 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-sys\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539414 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-systemd-units\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539440 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-etc-kubernetes\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539464 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-var-lib-openvswitch\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539476 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-host\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539487 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-etc-openvswitch\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539518 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-etc-kubernetes\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539519 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/588747e3-bc35-4a33-9545-3c606ef41faa-cni-binary-copy\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539531 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-node-log\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539556 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-multus-cni-dir\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539557 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-systemd-units\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.540078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539589 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-multus-socket-dir-parent\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539589 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-var-lib-openvswitch\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539439 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-sysconfig\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539625 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-device-dir\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539651 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-run-netns\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539665 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-node-log\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539676 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-run-k8s-cni-cncf-io\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539663 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-etc-openvswitch\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539695 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-multus-cni-dir\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539700 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-tmp\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539707 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/588747e3-bc35-4a33-9545-3c606ef41faa-multus-daemon-config\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539716 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-device-dir\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539717 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-multus-socket-dir-parent\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539733 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-run-netns\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539740 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-run-k8s-cni-cncf-io\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539766 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f309e28c-ca0c-40e2-b6b7-3beb2f3213c7-konnectivity-ca\") pod \"konnectivity-agent-2tz6j\" (UID: \"f309e28c-ca0c-40e2-b6b7-3beb2f3213c7\") " pod="kube-system/konnectivity-agent-2tz6j" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539695 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/af672f7a-0702-4a9b-8a25-5cab8441f3c2-dbus\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539794 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f558fde8-d070-4538-bb97-19bb59832b85-ovnkube-script-lib\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.540938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539818 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-var-lib-cni-multus\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539870 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-run-netns\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539887 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-run-multus-certs\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539912 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-run\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539942 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-run-netns\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539955 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh9qd\" (UniqueName: \"kubernetes.io/projected/0c6a2e31-247f-4534-845e-8e68feecb206-kube-api-access-dh9qd\") pod \"node-resolver-w2k5w\" (UID: \"0c6a2e31-247f-4534-845e-8e68feecb206\") " pod="openshift-dns/node-resolver-w2k5w" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539972 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-run-multus-certs\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.539980 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-cnibin\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540006 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-run\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540019 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-registration-dir\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540027 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-cnibin\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540048 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-kubelet\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540051 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-var-lib-cni-multus\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540075 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-run-ovn-kubernetes\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540080 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-registration-dir\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540093 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-kubelet\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540100 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f558fde8-d070-4538-bb97-19bb59832b85-ovn-node-metrics-cert\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540125 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-run-ovn-kubernetes\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.541770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540133 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-var-lib-cni-bin\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540162 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2r7f\" (UniqueName: \"kubernetes.io/projected/588747e3-bc35-4a33-9545-3c606ef41faa-kube-api-access-t2r7f\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540240 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-host-var-lib-cni-bin\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540272 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc0c8edc-89e0-4448-896c-78aa56a57d1c-host\") pod \"node-ca-66qw8\" (UID: \"dc0c8edc-89e0-4448-896c-78aa56a57d1c\") " pod="openshift-image-registry/node-ca-66qw8" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540297 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-systemd\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540322 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540350 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-cni-bin\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540367 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-systemd\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540377 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-var-lib-kubelet\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540389 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f558fde8-d070-4538-bb97-19bb59832b85-ovnkube-script-lib\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540405 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0c6a2e31-247f-4534-845e-8e68feecb206-hosts-file\") pod \"node-resolver-w2k5w\" (UID: \"0c6a2e31-247f-4534-845e-8e68feecb206\") " pod="openshift-dns/node-resolver-w2k5w" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540418 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-cni-bin\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540460 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-var-lib-kubelet\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540463 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0c6a2e31-247f-4534-845e-8e68feecb206-hosts-file\") pod \"node-resolver-w2k5w\" (UID: \"0c6a2e31-247f-4534-845e-8e68feecb206\") " pod="openshift-dns/node-resolver-w2k5w" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:57.540487 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540504 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-hostroot\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540527 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540541 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-hostroot\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.542447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540544 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-sysctl-d\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540587 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-run-ovn\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:57.540611 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret podName:af672f7a-0702-4a9b-8a25-5cab8441f3c2 nodeName:}" failed. No retries permitted until 2026-09-10 17:39:58.040589137 +0000 UTC m=+2.104999840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret") pod "global-pull-secret-syncer-s42t8" (UID: "af672f7a-0702-4a9b-8a25-5cab8441f3c2") : object "kube-system"/"original-pull-secret" not registered Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540612 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540624 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-run-ovn\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540646 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f558fde8-d070-4538-bb97-19bb59832b85-ovnkube-config\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540664 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-sysctl-d\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540676 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crgj7\" (UniqueName: \"kubernetes.io/projected/f558fde8-d070-4538-bb97-19bb59832b85-kube-api-access-crgj7\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540718 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs\") pod \"network-metrics-daemon-cvx9n\" (UID: \"b563ccdb-296e-4358-81f7-82dc6bd66435\") " pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540745 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-multus-conf-dir\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540771 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540797 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-sys-fs\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540824 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-log-socket\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540852 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:57.540862 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f558fde8-d070-4538-bb97-19bb59832b85-env-overrides\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540909 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f309e28c-ca0c-40e2-b6b7-3beb2f3213c7-konnectivity-ca\") pod \"konnectivity-agent-2tz6j\" (UID: \"f309e28c-ca0c-40e2-b6b7-3beb2f3213c7\") " pod="kube-system/konnectivity-agent-2tz6j" Sep 10 17:39:57.543043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540909 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c6a2e31-247f-4534-845e-8e68feecb206-tmp-dir\") pod \"node-resolver-w2k5w\" (UID: \"0c6a2e31-247f-4534-845e-8e68feecb206\") " pod="openshift-dns/node-resolver-w2k5w" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:57.540945 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs podName:b563ccdb-296e-4358-81f7-82dc6bd66435 nodeName:}" failed. No retries permitted until 2026-09-10 17:39:58.040906203 +0000 UTC m=+2.105316901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs") pod "network-metrics-daemon-cvx9n" (UID: "b563ccdb-296e-4358-81f7-82dc6bd66435") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540970 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-sys-fs\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540987 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-modprobe-d\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.540990 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541018 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-kubernetes\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541025 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/588747e3-bc35-4a33-9545-3c606ef41faa-multus-conf-dir\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541039 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-log-socket\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541058 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f309e28c-ca0c-40e2-b6b7-3beb2f3213c7-agent-certs\") pod \"konnectivity-agent-2tz6j\" (UID: \"f309e28c-ca0c-40e2-b6b7-3beb2f3213c7\") " pod="kube-system/konnectivity-agent-2tz6j" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541087 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-run-openvswitch\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541091 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-kubernetes\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541115 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-modprobe-d\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541117 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jctb\" (UniqueName: \"kubernetes.io/projected/b563ccdb-296e-4358-81f7-82dc6bd66435-kube-api-access-4jctb\") pod \"network-metrics-daemon-cvx9n\" (UID: \"b563ccdb-296e-4358-81f7-82dc6bd66435\") " pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541149 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f558fde8-d070-4538-bb97-19bb59832b85-ovnkube-config\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541153 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggcw7\" (UniqueName: \"kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7\") pod \"network-check-target-5k8rp\" (UID: \"c3d44764-8e31-4aaf-a532-86de76fc6820\") " pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541196 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-slash\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541210 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c6a2e31-247f-4534-845e-8e68feecb206-tmp-dir\") pod \"node-resolver-w2k5w\" (UID: \"0c6a2e31-247f-4534-845e-8e68feecb206\") " pod="openshift-dns/node-resolver-w2k5w" Sep 10 17:39:57.543599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541264 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-host-slash\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.544138 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541309 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f558fde8-d070-4538-bb97-19bb59832b85-run-openvswitch\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.544138 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541345 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc0c8edc-89e0-4448-896c-78aa56a57d1c-host\") pod \"node-ca-66qw8\" (UID: \"dc0c8edc-89e0-4448-896c-78aa56a57d1c\") " pod="openshift-image-registry/node-ca-66qw8" Sep 10 17:39:57.544138 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541389 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/88785664-b748-45f4-a3bd-c6baffa832b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.544138 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.541446 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f558fde8-d070-4538-bb97-19bb59832b85-env-overrides\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.544138 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.542281 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-tmp\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.544138 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.542319 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-etc-tuned\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.544138 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.543258 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f558fde8-d070-4538-bb97-19bb59832b85-ovn-node-metrics-cert\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.544138 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.544074 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f309e28c-ca0c-40e2-b6b7-3beb2f3213c7-agent-certs\") pod \"konnectivity-agent-2tz6j\" (UID: \"f309e28c-ca0c-40e2-b6b7-3beb2f3213c7\") " pod="kube-system/konnectivity-agent-2tz6j" Sep 10 17:39:57.559603 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.559571 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvbsg\" (UniqueName: \"kubernetes.io/projected/dc0c8edc-89e0-4448-896c-78aa56a57d1c-kube-api-access-kvbsg\") pod \"node-ca-66qw8\" (UID: \"dc0c8edc-89e0-4448-896c-78aa56a57d1c\") " pod="openshift-image-registry/node-ca-66qw8" Sep 10 17:39:57.561168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.561147 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8wzk\" (UniqueName: \"kubernetes.io/projected/88785664-b748-45f4-a3bd-c6baffa832b0-kube-api-access-p8wzk\") pod \"aws-ebs-csi-driver-node-t252r\" (UID: \"88785664-b748-45f4-a3bd-c6baffa832b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.578164 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:57.578141 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 17:39:57.578164 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:57.578162 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 17:39:57.578285 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:57.578176 2565 projected.go:196] Error preparing data for projected volume kube-api-access-ggcw7 for pod openshift-network-diagnostics/network-check-target-5k8rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:39:57.578285 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:57.578231 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7 podName:c3d44764-8e31-4aaf-a532-86de76fc6820 nodeName:}" failed. No retries permitted until 2026-09-10 17:39:58.078215781 +0000 UTC m=+2.142626495 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ggcw7" (UniqueName: "kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7") pod "network-check-target-5k8rp" (UID: "c3d44764-8e31-4aaf-a532-86de76fc6820") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:39:57.579875 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.579855 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mltlv\" (UniqueName: \"kubernetes.io/projected/38a5bec0-a6f4-4371-b852-9b3e38a2ae47-kube-api-access-mltlv\") pod \"tuned-vgrm9\" (UID: \"38a5bec0-a6f4-4371-b852-9b3e38a2ae47\") " pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.581779 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.581755 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2r7f\" (UniqueName: \"kubernetes.io/projected/588747e3-bc35-4a33-9545-3c606ef41faa-kube-api-access-t2r7f\") pod \"multus-nj8jg\" (UID: \"588747e3-bc35-4a33-9545-3c606ef41faa\") " pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.584200 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.584182 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh9qd\" (UniqueName: \"kubernetes.io/projected/0c6a2e31-247f-4534-845e-8e68feecb206-kube-api-access-dh9qd\") pod \"node-resolver-w2k5w\" (UID: \"0c6a2e31-247f-4534-845e-8e68feecb206\") " pod="openshift-dns/node-resolver-w2k5w" Sep 10 17:39:57.587407 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.587385 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crgj7\" (UniqueName: \"kubernetes.io/projected/f558fde8-d070-4538-bb97-19bb59832b85-kube-api-access-crgj7\") pod \"ovnkube-node-g2pbs\" (UID: \"f558fde8-d070-4538-bb97-19bb59832b85\") " pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.597846 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.597825 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jctb\" (UniqueName: \"kubernetes.io/projected/b563ccdb-296e-4358-81f7-82dc6bd66435-kube-api-access-4jctb\") pod \"network-metrics-daemon-cvx9n\" (UID: \"b563ccdb-296e-4358-81f7-82dc6bd66435\") " pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:39:57.643962 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.643930 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 17:39:57.649988 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.649963 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-r4tz2" Sep 10 17:39:57.661476 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:57.661449 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3708abb_21c9_4590_90a9_00cfec61f382.slice/crio-c1d08c7190391a9f6f42d167e78d6a5141c5a57790a54b427867624c43d14828 WatchSource:0}: Error finding container c1d08c7190391a9f6f42d167e78d6a5141c5a57790a54b427867624c43d14828: Status 404 returned error can't find the container with id c1d08c7190391a9f6f42d167e78d6a5141c5a57790a54b427867624c43d14828 Sep 10 17:39:57.665047 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.665028 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xnch8" Sep 10 17:39:57.673089 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:57.673066 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df73191_ea82_4144_83bf_57c0d770a4a2.slice/crio-2fddfa7aa993f5a38e587bf3c1f736998bf3af7c35f98e416a9d297753dcd356 WatchSource:0}: Error finding container 2fddfa7aa993f5a38e587bf3c1f736998bf3af7c35f98e416a9d297753dcd356: Status 404 returned error can't find the container with id 2fddfa7aa993f5a38e587bf3c1f736998bf3af7c35f98e416a9d297753dcd356 Sep 10 17:39:57.680446 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.680428 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:39:57.688643 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:57.688624 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf558fde8_d070_4538_bb97_19bb59832b85.slice/crio-d78293b6d42b9f954f137391574ee636dd9c2cc5188bb327d32d13ae41a058d3 WatchSource:0}: Error finding container d78293b6d42b9f954f137391574ee636dd9c2cc5188bb327d32d13ae41a058d3: Status 404 returned error can't find the container with id d78293b6d42b9f954f137391574ee636dd9c2cc5188bb327d32d13ae41a058d3 Sep 10 17:39:57.700383 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.700366 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" Sep 10 17:39:57.708788 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:57.708760 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88785664_b748_45f4_a3bd_c6baffa832b0.slice/crio-f7004b7993c319bdcbcc8234db42041758f3574b605803f577ae51eebf90033e WatchSource:0}: Error finding container f7004b7993c319bdcbcc8234db42041758f3574b605803f577ae51eebf90033e: Status 404 returned error can't find the container with id f7004b7993c319bdcbcc8234db42041758f3574b605803f577ae51eebf90033e Sep 10 17:39:57.715474 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.715457 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w2k5w" Sep 10 17:39:57.721331 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.721310 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-66qw8" Sep 10 17:39:57.723280 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:57.723260 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c6a2e31_247f_4534_845e_8e68feecb206.slice/crio-0d1c6001a61ad12238545ed64dd67b47a7ab679d2ecc3622f6c010f2005f3a92 WatchSource:0}: Error finding container 0d1c6001a61ad12238545ed64dd67b47a7ab679d2ecc3622f6c010f2005f3a92: Status 404 returned error can't find the container with id 0d1c6001a61ad12238545ed64dd67b47a7ab679d2ecc3622f6c010f2005f3a92 Sep 10 17:39:57.729954 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:57.729935 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc0c8edc_89e0_4448_896c_78aa56a57d1c.slice/crio-53be0e35d3bcef5223ad5eb1f44b01337162d9245c1e634c39d58c56d54298b6 WatchSource:0}: Error finding container 53be0e35d3bcef5223ad5eb1f44b01337162d9245c1e634c39d58c56d54298b6: Status 404 returned error can't find the container with id 53be0e35d3bcef5223ad5eb1f44b01337162d9245c1e634c39d58c56d54298b6 Sep 10 17:39:57.734238 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.734214 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" Sep 10 17:39:57.742877 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:57.742860 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a5bec0_a6f4_4371_b852_9b3e38a2ae47.slice/crio-053d5a8e4ec00dc14a3e2105ff95db8672c2c37fd5189dbcb6f57a224a336300 WatchSource:0}: Error finding container 053d5a8e4ec00dc14a3e2105ff95db8672c2c37fd5189dbcb6f57a224a336300: Status 404 returned error can't find the container with id 053d5a8e4ec00dc14a3e2105ff95db8672c2c37fd5189dbcb6f57a224a336300 Sep 10 17:39:57.751696 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.751678 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2tz6j" Sep 10 17:39:57.759601 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:57.759582 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf309e28c_ca0c_40e2_b6b7_3beb2f3213c7.slice/crio-04f8f24201e5910660a7e564adaa3a156a30d594d37ff0d874517d044845dd27 WatchSource:0}: Error finding container 04f8f24201e5910660a7e564adaa3a156a30d594d37ff0d874517d044845dd27: Status 404 returned error can't find the container with id 04f8f24201e5910660a7e564adaa3a156a30d594d37ff0d874517d044845dd27 Sep 10 17:39:57.767442 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.767424 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nj8jg" Sep 10 17:39:57.776551 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:39:57.776530 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod588747e3_bc35_4a33_9545_3c606ef41faa.slice/crio-e2984042bc8033c1e90d974305371f566c139156b1fc5f282446132deed520df WatchSource:0}: Error finding container e2984042bc8033c1e90d974305371f566c139156b1fc5f282446132deed520df: Status 404 returned error can't find the container with id e2984042bc8033c1e90d974305371f566c139156b1fc5f282446132deed520df Sep 10 17:39:57.875152 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:57.875128 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 17:39:58.045602 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.045254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs\") pod \"network-metrics-daemon-cvx9n\" (UID: \"b563ccdb-296e-4358-81f7-82dc6bd66435\") " pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:39:58.045602 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.045365 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:39:58.045602 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:58.045416 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:39:58.045602 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:58.045484 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 17:39:58.045602 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:58.045500 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs podName:b563ccdb-296e-4358-81f7-82dc6bd66435 nodeName:}" failed. No retries permitted until 2026-09-10 17:39:59.045484676 +0000 UTC m=+3.109895366 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs") pod "network-metrics-daemon-cvx9n" (UID: "b563ccdb-296e-4358-81f7-82dc6bd66435") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:39:58.045602 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:58.045532 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret podName:af672f7a-0702-4a9b-8a25-5cab8441f3c2 nodeName:}" failed. No retries permitted until 2026-09-10 17:39:59.045518182 +0000 UTC m=+3.109928873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret") pod "global-pull-secret-syncer-s42t8" (UID: "af672f7a-0702-4a9b-8a25-5cab8441f3c2") : object "kube-system"/"original-pull-secret" not registered Sep 10 17:39:58.146179 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.146020 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggcw7\" (UniqueName: \"kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7\") pod \"network-check-target-5k8rp\" (UID: \"c3d44764-8e31-4aaf-a532-86de76fc6820\") " pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:39:58.146365 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:58.146254 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 17:39:58.146365 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:58.146290 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 17:39:58.146365 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:58.146306 2565 projected.go:196] Error preparing data for projected volume kube-api-access-ggcw7 for pod openshift-network-diagnostics/network-check-target-5k8rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:39:58.146503 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:58.146394 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7 podName:c3d44764-8e31-4aaf-a532-86de76fc6820 nodeName:}" failed. No retries permitted until 2026-09-10 17:39:59.146373118 +0000 UTC m=+3.210783814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggcw7" (UniqueName: "kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7") pod "network-check-target-5k8rp" (UID: "c3d44764-8e31-4aaf-a532-86de76fc6820") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:39:58.381395 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.381268 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 17:34:58 +0000 UTC" deadline="2028-02-05 14:15:48.21611059 +0000 UTC" Sep 10 17:39:58.381395 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.381337 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12308h35m49.834778566s" Sep 10 17:39:58.466226 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.466103 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4tz2" event={"ID":"e3708abb-21c9-4590-90a9-00cfec61f382","Type":"ContainerStarted","Data":"c1d08c7190391a9f6f42d167e78d6a5141c5a57790a54b427867624c43d14828"} Sep 10 17:39:58.477481 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.477383 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal" event={"ID":"3c069671a249c5c5f8caa70b379bea25","Type":"ContainerStarted","Data":"cc739e937766c20c00f46b455367a2399261683eb1434a5b509be3806e1d4b60"} Sep 10 17:39:58.481763 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.481731 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nj8jg" event={"ID":"588747e3-bc35-4a33-9545-3c606ef41faa","Type":"ContainerStarted","Data":"e2984042bc8033c1e90d974305371f566c139156b1fc5f282446132deed520df"} Sep 10 17:39:58.488652 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.488611 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-66qw8" event={"ID":"dc0c8edc-89e0-4448-896c-78aa56a57d1c","Type":"ContainerStarted","Data":"53be0e35d3bcef5223ad5eb1f44b01337162d9245c1e634c39d58c56d54298b6"} Sep 10 17:39:58.493111 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.493084 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w2k5w" event={"ID":"0c6a2e31-247f-4534-845e-8e68feecb206","Type":"ContainerStarted","Data":"0d1c6001a61ad12238545ed64dd67b47a7ab679d2ecc3622f6c010f2005f3a92"} Sep 10 17:39:58.505363 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.505333 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" event={"ID":"88785664-b748-45f4-a3bd-c6baffa832b0","Type":"ContainerStarted","Data":"f7004b7993c319bdcbcc8234db42041758f3574b605803f577ae51eebf90033e"} Sep 10 17:39:58.523520 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.523482 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" event={"ID":"f558fde8-d070-4538-bb97-19bb59832b85","Type":"ContainerStarted","Data":"d78293b6d42b9f954f137391574ee636dd9c2cc5188bb327d32d13ae41a058d3"} Sep 10 17:39:58.534198 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.534162 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xnch8" event={"ID":"5df73191-ea82-4144-83bf-57c0d770a4a2","Type":"ContainerStarted","Data":"2fddfa7aa993f5a38e587bf3c1f736998bf3af7c35f98e416a9d297753dcd356"} Sep 10 17:39:58.548008 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.547974 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-84.ec2.internal" event={"ID":"b0f5917d8bc06c28727df72b1ab49ae9","Type":"ContainerStarted","Data":"2bf580594af14cd7e61adbbe323aaed54a59703b70ca44e61cdff8c422f25706"} Sep 10 17:39:58.560638 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.560610 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2tz6j" event={"ID":"f309e28c-ca0c-40e2-b6b7-3beb2f3213c7","Type":"ContainerStarted","Data":"04f8f24201e5910660a7e564adaa3a156a30d594d37ff0d874517d044845dd27"} Sep 10 17:39:58.584448 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.584418 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" event={"ID":"38a5bec0-a6f4-4371-b852-9b3e38a2ae47","Type":"ContainerStarted","Data":"053d5a8e4ec00dc14a3e2105ff95db8672c2c37fd5189dbcb6f57a224a336300"} Sep 10 17:39:58.671606 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:58.671522 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 17:39:59.053712 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:59.053625 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs\") pod \"network-metrics-daemon-cvx9n\" (UID: \"b563ccdb-296e-4358-81f7-82dc6bd66435\") " pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:39:59.053893 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:59.053730 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:39:59.053893 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:59.053852 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 17:39:59.054056 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:59.053915 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret podName:af672f7a-0702-4a9b-8a25-5cab8441f3c2 nodeName:}" failed. No retries permitted until 2026-09-10 17:40:01.053897418 +0000 UTC m=+5.118308114 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret") pod "global-pull-secret-syncer-s42t8" (UID: "af672f7a-0702-4a9b-8a25-5cab8441f3c2") : object "kube-system"/"original-pull-secret" not registered Sep 10 17:39:59.054391 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:59.054340 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:39:59.054475 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:59.054396 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs podName:b563ccdb-296e-4358-81f7-82dc6bd66435 nodeName:}" failed. No retries permitted until 2026-09-10 17:40:01.054380572 +0000 UTC m=+5.118791278 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs") pod "network-metrics-daemon-cvx9n" (UID: "b563ccdb-296e-4358-81f7-82dc6bd66435") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:39:59.158055 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:59.158010 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggcw7\" (UniqueName: \"kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7\") pod \"network-check-target-5k8rp\" (UID: \"c3d44764-8e31-4aaf-a532-86de76fc6820\") " pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:39:59.158252 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:59.158231 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 17:39:59.158321 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:59.158263 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 17:39:59.158321 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:59.158279 2565 projected.go:196] Error preparing data for projected volume kube-api-access-ggcw7 for pod openshift-network-diagnostics/network-check-target-5k8rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:39:59.158435 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:59.158348 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7 podName:c3d44764-8e31-4aaf-a532-86de76fc6820 nodeName:}" failed. No retries permitted until 2026-09-10 17:40:01.158325255 +0000 UTC m=+5.222735969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggcw7" (UniqueName: "kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7") pod "network-check-target-5k8rp" (UID: "c3d44764-8e31-4aaf-a532-86de76fc6820") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:39:59.382332 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:59.382209 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 17:34:58 +0000 UTC" deadline="2028-05-12 22:51:31.733575002 +0000 UTC" Sep 10 17:39:59.382332 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:59.382248 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14645h11m32.351332011s" Sep 10 17:39:59.433245 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:59.432333 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:39:59.433245 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:59.432479 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:39:59.433245 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:59.432922 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:39:59.433245 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:59.433049 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:39:59.433245 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:39:59.433118 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:39:59.433245 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:39:59.433193 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:00.363794 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:00.363563 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 17:40:01.083449 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:01.083410 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:01.084119 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:01.083469 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs\") pod \"network-metrics-daemon-cvx9n\" (UID: \"b563ccdb-296e-4358-81f7-82dc6bd66435\") " pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:01.084119 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:01.083630 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:40:01.084119 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:01.083699 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs podName:b563ccdb-296e-4358-81f7-82dc6bd66435 nodeName:}" failed. No retries permitted until 2026-09-10 17:40:05.08367828 +0000 UTC m=+9.148088971 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs") pod "network-metrics-daemon-cvx9n" (UID: "b563ccdb-296e-4358-81f7-82dc6bd66435") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:40:01.084119 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:01.083630 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 17:40:01.084119 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:01.083795 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret podName:af672f7a-0702-4a9b-8a25-5cab8441f3c2 nodeName:}" failed. No retries permitted until 2026-09-10 17:40:05.083774444 +0000 UTC m=+9.148185151 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret") pod "global-pull-secret-syncer-s42t8" (UID: "af672f7a-0702-4a9b-8a25-5cab8441f3c2") : object "kube-system"/"original-pull-secret" not registered Sep 10 17:40:01.184224 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:01.184179 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggcw7\" (UniqueName: \"kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7\") pod \"network-check-target-5k8rp\" (UID: \"c3d44764-8e31-4aaf-a532-86de76fc6820\") " pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:01.184394 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:01.184369 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 17:40:01.184394 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:01.184393 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 17:40:01.184517 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:01.184406 2565 projected.go:196] Error preparing data for projected volume kube-api-access-ggcw7 for pod openshift-network-diagnostics/network-check-target-5k8rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:40:01.184517 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:01.184481 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7 podName:c3d44764-8e31-4aaf-a532-86de76fc6820 nodeName:}" failed. No retries permitted until 2026-09-10 17:40:05.184460814 +0000 UTC m=+9.248871531 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggcw7" (UniqueName: "kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7") pod "network-check-target-5k8rp" (UID: "c3d44764-8e31-4aaf-a532-86de76fc6820") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:40:01.431804 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:01.431756 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:01.432015 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:01.431923 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:01.432390 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:01.432369 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:01.432519 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:01.432477 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:01.432580 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:01.432533 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:01.432637 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:01.432624 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:03.432456 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:03.432349 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:03.432905 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:03.432485 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:03.432905 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:03.432503 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:03.432905 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:03.432638 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:03.432905 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:03.432675 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:03.432905 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:03.432767 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:05.119966 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:05.119898 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:05.120461 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:05.119989 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs\") pod \"network-metrics-daemon-cvx9n\" (UID: \"b563ccdb-296e-4358-81f7-82dc6bd66435\") " pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:05.120461 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:05.120110 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 17:40:05.120461 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:05.120151 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:40:05.120461 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:05.120202 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret podName:af672f7a-0702-4a9b-8a25-5cab8441f3c2 nodeName:}" failed. No retries permitted until 2026-09-10 17:40:13.120179544 +0000 UTC m=+17.184590249 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret") pod "global-pull-secret-syncer-s42t8" (UID: "af672f7a-0702-4a9b-8a25-5cab8441f3c2") : object "kube-system"/"original-pull-secret" not registered Sep 10 17:40:05.120461 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:05.120228 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs podName:b563ccdb-296e-4358-81f7-82dc6bd66435 nodeName:}" failed. No retries permitted until 2026-09-10 17:40:13.120215111 +0000 UTC m=+17.184625805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs") pod "network-metrics-daemon-cvx9n" (UID: "b563ccdb-296e-4358-81f7-82dc6bd66435") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:40:05.221024 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:05.220981 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggcw7\" (UniqueName: \"kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7\") pod \"network-check-target-5k8rp\" (UID: \"c3d44764-8e31-4aaf-a532-86de76fc6820\") " pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:05.221237 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:05.221147 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 17:40:05.221237 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:05.221175 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 17:40:05.221237 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:05.221189 2565 projected.go:196] Error preparing data for projected volume kube-api-access-ggcw7 for pod openshift-network-diagnostics/network-check-target-5k8rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:40:05.221375 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:05.221257 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7 podName:c3d44764-8e31-4aaf-a532-86de76fc6820 nodeName:}" failed. No retries permitted until 2026-09-10 17:40:13.221236539 +0000 UTC m=+17.285647237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggcw7" (UniqueName: "kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7") pod "network-check-target-5k8rp" (UID: "c3d44764-8e31-4aaf-a532-86de76fc6820") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:40:05.432112 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:05.431791 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:05.432112 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:05.431826 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:05.432112 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:05.431826 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:05.432112 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:05.431954 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:05.432112 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:05.432109 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:05.432503 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:05.432244 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:07.432081 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:07.432035 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:07.432634 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:07.432035 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:07.432634 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:07.432186 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:07.432634 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:07.432227 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:07.432634 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:07.432331 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:07.432634 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:07.432425 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:09.432055 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:09.432017 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:09.432055 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:09.432036 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:09.432055 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:09.432053 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:09.432463 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:09.432128 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:09.432463 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:09.432247 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:09.432463 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:09.432306 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:11.431570 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:11.431534 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:11.431570 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:11.431566 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:11.432066 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:11.431540 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:11.432066 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:11.431671 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:11.432066 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:11.431731 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:11.432066 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:11.431809 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:13.179500 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:13.179464 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs\") pod \"network-metrics-daemon-cvx9n\" (UID: \"b563ccdb-296e-4358-81f7-82dc6bd66435\") " pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:13.179954 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:13.179575 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:13.179954 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:13.179632 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:40:13.179954 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:13.179687 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 17:40:13.179954 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:13.179713 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs podName:b563ccdb-296e-4358-81f7-82dc6bd66435 nodeName:}" failed. No retries permitted until 2026-09-10 17:40:29.179692678 +0000 UTC m=+33.244103373 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs") pod "network-metrics-daemon-cvx9n" (UID: "b563ccdb-296e-4358-81f7-82dc6bd66435") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:40:13.179954 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:13.179755 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret podName:af672f7a-0702-4a9b-8a25-5cab8441f3c2 nodeName:}" failed. No retries permitted until 2026-09-10 17:40:29.179729519 +0000 UTC m=+33.244140212 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret") pod "global-pull-secret-syncer-s42t8" (UID: "af672f7a-0702-4a9b-8a25-5cab8441f3c2") : object "kube-system"/"original-pull-secret" not registered Sep 10 17:40:13.280180 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:13.280142 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggcw7\" (UniqueName: \"kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7\") pod \"network-check-target-5k8rp\" (UID: \"c3d44764-8e31-4aaf-a532-86de76fc6820\") " pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:13.280335 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:13.280302 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 17:40:13.280335 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:13.280317 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 17:40:13.280335 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:13.280327 2565 projected.go:196] Error preparing data for projected volume kube-api-access-ggcw7 for pod openshift-network-diagnostics/network-check-target-5k8rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:40:13.280491 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:13.280387 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7 podName:c3d44764-8e31-4aaf-a532-86de76fc6820 nodeName:}" failed. No retries permitted until 2026-09-10 17:40:29.280372735 +0000 UTC m=+33.344783424 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggcw7" (UniqueName: "kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7") pod "network-check-target-5k8rp" (UID: "c3d44764-8e31-4aaf-a532-86de76fc6820") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:40:13.431623 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:13.431534 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:13.431770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:13.431556 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:13.431770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:13.431651 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:13.431770 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:13.431659 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:13.431770 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:13.431736 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:13.432029 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:13.431879 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:15.431705 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:15.431670 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:15.432140 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:15.431669 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:15.432140 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:15.431788 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:15.432140 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:15.431669 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:15.432140 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:15.431872 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:15.432140 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:15.431944 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:17.431509 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.431260 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:17.431963 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:17.431613 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:17.431963 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.431390 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:17.431963 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:17.431713 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:17.431963 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.431266 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:17.431963 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:17.431805 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:17.649318 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.649064 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g2pbs_f558fde8-d070-4538-bb97-19bb59832b85/ovn-acl-logging/0.log" Sep 10 17:40:17.651073 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.651038 2565 generic.go:356] "Generic (PLEG): container finished" podID="f558fde8-d070-4538-bb97-19bb59832b85" containerID="076ddce0a4fe8c0042b61571b3dc51fe39ed65ff450bd76181466012da90848e" exitCode=1 Sep 10 17:40:17.651180 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.651142 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" event={"ID":"f558fde8-d070-4538-bb97-19bb59832b85","Type":"ContainerStarted","Data":"1fd0cbad899eaa6e9c3d9e4e2adc507722b8c30d25220235f3b2737d10381044"} Sep 10 17:40:17.651246 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.651228 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" event={"ID":"f558fde8-d070-4538-bb97-19bb59832b85","Type":"ContainerStarted","Data":"058e4d0aef231ba515c1a6e5b710be2458343d5f8a58115e48e39af1a1f895ba"} Sep 10 17:40:17.651295 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.651284 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" event={"ID":"f558fde8-d070-4538-bb97-19bb59832b85","Type":"ContainerStarted","Data":"bcfddfba4c7f75556abb5b0292ab37e0e72fa5a832022b039a85759104a3139f"} Sep 10 17:40:17.651357 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.651330 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" event={"ID":"f558fde8-d070-4538-bb97-19bb59832b85","Type":"ContainerDied","Data":"076ddce0a4fe8c0042b61571b3dc51fe39ed65ff450bd76181466012da90848e"} Sep 10 17:40:17.651409 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.651382 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" event={"ID":"f558fde8-d070-4538-bb97-19bb59832b85","Type":"ContainerStarted","Data":"6168d455544606a516454e4fd6e595ada387c6926ac23011b713d0b3ee2e5ecd"} Sep 10 17:40:17.654689 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.654644 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-84.ec2.internal" event={"ID":"b0f5917d8bc06c28727df72b1ab49ae9","Type":"ContainerStarted","Data":"22fe06af1e39c259b3eefcf5a3718acfecf99d9a1a393e9d3e2630aa8ace3f8f"} Sep 10 17:40:17.663168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.663135 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" event={"ID":"38a5bec0-a6f4-4371-b852-9b3e38a2ae47","Type":"ContainerStarted","Data":"63e17dfd28c8422fa40ce9bd0db7943dc15a31b7eba09a734d2f6424ce69c676"} Sep 10 17:40:17.668197 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.668142 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-84.ec2.internal" podStartSLOduration=19.66812607 podStartE2EDuration="19.66812607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:39:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:17.667919046 +0000 UTC m=+21.732329774" watchObservedRunningTime="2026-09-10 17:40:17.66812607 +0000 UTC m=+21.732536783" Sep 10 17:40:17.669814 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.669784 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nj8jg" event={"ID":"588747e3-bc35-4a33-9545-3c606ef41faa","Type":"ContainerStarted","Data":"a3cfadf2b786c24600a3786a82de6913c22b51cdf31a03eec0952edd124fc326"} Sep 10 17:40:17.682996 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.682940 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vgrm9" podStartSLOduration=1.360515141 podStartE2EDuration="20.682924934s" totalImagesPullingTime="19.322409793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:17.682732369 +0000 UTC m=+21.747143103" watchObservedRunningTime="2026-09-10 17:40:17.682924934 +0000 UTC m=+21.747335654" Sep 10 17:40:17.698571 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:17.698521 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nj8jg" podStartSLOduration=1.100091677 podStartE2EDuration="20.698502718s" totalImagesPullingTime="19.598411041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:17.69838544 +0000 UTC m=+21.762796165" watchObservedRunningTime="2026-09-10 17:40:17.698502718 +0000 UTC m=+21.762913426" Sep 10 17:40:18.674103 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.674061 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w2k5w" event={"ID":"0c6a2e31-247f-4534-845e-8e68feecb206","Type":"ContainerStarted","Data":"bce0114bdaf973478240b034c7b9d6ab2f087aa3a061a23b9eb390b0ac9ccefb"} Sep 10 17:40:18.675940 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.675845 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" event={"ID":"88785664-b748-45f4-a3bd-c6baffa832b0","Type":"ContainerStarted","Data":"35857225f39a8c28b59e15be2cbb04049907c46cd0008782a8b9be8f378dca87"} Sep 10 17:40:18.679413 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.679393 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g2pbs_f558fde8-d070-4538-bb97-19bb59832b85/ovn-acl-logging/0.log" Sep 10 17:40:18.679854 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.679827 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" event={"ID":"f558fde8-d070-4538-bb97-19bb59832b85","Type":"ContainerStarted","Data":"0aff1c47e27f4e72a0b288d7e753e3e2c6bca6f41e3d4d7d3bb19452cb1f4e38"} Sep 10 17:40:18.681650 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.681624 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xnch8" event={"ID":"5df73191-ea82-4144-83bf-57c0d770a4a2","Type":"ContainerStarted","Data":"4883ef40b7ea727eca8f600ba718285fe4f2561e7d80e0610d3191d1b3ddcd4d"} Sep 10 17:40:18.683388 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.683365 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2tz6j" event={"ID":"f309e28c-ca0c-40e2-b6b7-3beb2f3213c7","Type":"ContainerStarted","Data":"b2b55088e623648d43ebab74be0f00044e37c217b1cef0efe8a4e94cff8a3d03"} Sep 10 17:40:18.686800 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.685752 2565 generic.go:356] "Generic (PLEG): container finished" podID="e3708abb-21c9-4590-90a9-00cfec61f382" containerID="2252774e3e7b21467ba3bc85e6c8d8e60e9fc49e98627ef6dab6ca618e9fc657" exitCode=0 Sep 10 17:40:18.686800 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.685825 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4tz2" event={"ID":"e3708abb-21c9-4590-90a9-00cfec61f382","Type":"ContainerDied","Data":"2252774e3e7b21467ba3bc85e6c8d8e60e9fc49e98627ef6dab6ca618e9fc657"} Sep 10 17:40:18.688721 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.688697 2565 generic.go:356] "Generic (PLEG): container finished" podID="3c069671a249c5c5f8caa70b379bea25" containerID="7f74087314af1dc74a4761e8a758e24902645003a4832c59738d46cce9d9dc75" exitCode=0 Sep 10 17:40:18.688814 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.688785 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal" event={"ID":"3c069671a249c5c5f8caa70b379bea25","Type":"ContainerDied","Data":"7f74087314af1dc74a4761e8a758e24902645003a4832c59738d46cce9d9dc75"} Sep 10 17:40:18.690324 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.690284 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-w2k5w" podStartSLOduration=2.350061045 podStartE2EDuration="21.690270133s" totalImagesPullingTime="19.340209088s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:18.689968042 +0000 UTC m=+22.754378770" watchObservedRunningTime="2026-09-10 17:40:18.690270133 +0000 UTC m=+22.754680844" Sep 10 17:40:18.690540 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.690518 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-66qw8" event={"ID":"dc0c8edc-89e0-4448-896c-78aa56a57d1c","Type":"ContainerStarted","Data":"09b9e1d568f5e7a6811dc65e6280f0afb288b47891bebcd90af3e2bb423fde4e"} Sep 10 17:40:18.713587 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.713545 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2tz6j" podStartSLOduration=2.40956215 podStartE2EDuration="21.713531086s" totalImagesPullingTime="19.303968936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:18.712930157 +0000 UTC m=+22.777340893" watchObservedRunningTime="2026-09-10 17:40:18.713531086 +0000 UTC m=+22.777941796" Sep 10 17:40:18.759468 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.759423 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xnch8" podStartSLOduration=2.36891967 podStartE2EDuration="21.759408843s" totalImagesPullingTime="19.390489173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:18.759084006 +0000 UTC m=+22.823494732" watchObservedRunningTime="2026-09-10 17:40:18.759408843 +0000 UTC m=+22.823819554" Sep 10 17:40:18.780397 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.780187 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-66qw8" podStartSLOduration=2.445966242 podStartE2EDuration="21.780172834s" totalImagesPullingTime="19.334206592s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:18.779730417 +0000 UTC m=+22.844141144" watchObservedRunningTime="2026-09-10 17:40:18.780172834 +0000 UTC m=+22.844583546" Sep 10 17:40:18.903048 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.903017 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:40:18.977852 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:18.977820 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:40:19.022491 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.022457 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2tz6j" Sep 10 17:40:19.023061 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.023045 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2tz6j" Sep 10 17:40:19.049010 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.048985 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 17:40:19.393397 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.393287 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T17:40:19.04900447Z","UUID":"f2b3110f-ea80-4f8e-8d4d-1f642d60dd92","Handler":null,"Name":"","Endpoint":""} Sep 10 17:40:19.395020 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.394988 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 17:40:19.395020 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.395025 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 17:40:19.431289 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.431260 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:19.431289 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.431273 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:19.431472 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.431273 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:19.431472 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:19.431346 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:19.431472 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:19.431436 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:19.431578 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:19.431505 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:19.694544 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.694459 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal" event={"ID":"3c069671a249c5c5f8caa70b379bea25","Type":"ContainerStarted","Data":"e417d499427885d5889f65d345ea599d4ee5cacd96e2fe92f97b42e9b47cf01d"} Sep 10 17:40:19.696328 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.696304 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" event={"ID":"88785664-b748-45f4-a3bd-c6baffa832b0","Type":"ContainerStarted","Data":"e9208c487bc8bd7ea4dfd31de5d1265b530c514ffce716293ad6317ce59a260d"} Sep 10 17:40:19.696735 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.696715 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2tz6j" Sep 10 17:40:19.697176 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.697161 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2tz6j" Sep 10 17:40:19.709327 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.709283 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-84.ec2.internal" podStartSLOduration=21.709271016 podStartE2EDuration="21.709271016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:39:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:19.708969312 +0000 UTC m=+23.773380036" watchObservedRunningTime="2026-09-10 17:40:19.709271016 +0000 UTC m=+23.773681727" Sep 10 17:40:19.770905 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:19.770856 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:40:20.701359 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:20.701296 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" event={"ID":"88785664-b748-45f4-a3bd-c6baffa832b0","Type":"ContainerStarted","Data":"2136928d57e5b322d46ea860b6da92fe46977bc750e3842aeca032e92021103f"} Sep 10 17:40:20.705208 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:20.705183 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g2pbs_f558fde8-d070-4538-bb97-19bb59832b85/ovn-acl-logging/0.log" Sep 10 17:40:20.705679 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:20.705656 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" event={"ID":"f558fde8-d070-4538-bb97-19bb59832b85","Type":"ContainerStarted","Data":"176612abf7f37c9622774bb09ed8cdd3d79e63486790c1c5a4bdce9057e0e2b2"} Sep 10 17:40:20.721497 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:20.721435 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t252r" podStartSLOduration=1.958883419 podStartE2EDuration="23.721420075s" totalImagesPullingTime="21.762536656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:39:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:20.720798339 +0000 UTC m=+24.785209072" watchObservedRunningTime="2026-09-10 17:40:20.721420075 +0000 UTC m=+24.785830786" Sep 10 17:40:21.431399 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:21.431368 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:21.431603 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:21.431368 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:21.431603 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:21.431468 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:21.431603 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:21.431368 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:21.431723 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:21.431539 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:21.431723 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:21.431644 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:22.714659 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:22.714471 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g2pbs_f558fde8-d070-4538-bb97-19bb59832b85/ovn-acl-logging/0.log" Sep 10 17:40:22.715163 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:22.715136 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" event={"ID":"f558fde8-d070-4538-bb97-19bb59832b85","Type":"ContainerStarted","Data":"8b946723751df992c7b9eb4dc07c04a2cc337ff94db22f4f5d1c062d938baa8e"} Sep 10 17:40:22.715493 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:22.715467 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:40:22.715575 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:22.715541 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:40:22.715728 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:22.715710 2565 scope.go:117] "RemoveContainer" containerID="076ddce0a4fe8c0042b61571b3dc51fe39ed65ff450bd76181466012da90848e" Sep 10 17:40:22.733422 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:22.733394 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:40:22.733542 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:22.733530 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:40:23.432909 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:23.432181 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:23.432909 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:23.432181 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:23.432909 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:23.432316 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:23.432909 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:23.432349 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:23.432909 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:23.432423 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:23.432909 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:23.432486 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:23.722104 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:23.722030 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g2pbs_f558fde8-d070-4538-bb97-19bb59832b85/ovn-acl-logging/0.log" Sep 10 17:40:23.722694 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:23.722487 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" event={"ID":"f558fde8-d070-4538-bb97-19bb59832b85","Type":"ContainerStarted","Data":"dd2f4a681704ebda94d85e18b2291804f4c9a383466d36f4a81eb52e614eaa6c"} Sep 10 17:40:23.722694 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:23.722597 2565 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 10 17:40:23.750852 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:23.750806 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" podStartSLOduration=7.343572314 podStartE2EDuration="26.750791235s" totalImagesPullingTime="19.407218921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:23.750021204 +0000 UTC m=+27.814431933" watchObservedRunningTime="2026-09-10 17:40:23.750791235 +0000 UTC m=+27.815201953" Sep 10 17:40:23.847249 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:23.847207 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:40:24.118054 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:24.117974 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s42t8"] Sep 10 17:40:24.118212 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:24.118099 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:24.118271 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:24.118215 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:24.121423 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:24.121391 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cvx9n"] Sep 10 17:40:24.121537 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:24.121490 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:24.121615 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:24.121592 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:24.122114 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:24.122092 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5k8rp"] Sep 10 17:40:24.122225 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:24.122172 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:24.122283 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:24.122252 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:25.431596 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:25.431554 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:25.432121 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:25.431675 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:25.728851 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:25.728816 2565 generic.go:356] "Generic (PLEG): container finished" podID="e3708abb-21c9-4590-90a9-00cfec61f382" containerID="505ce2f542ba6dd6ca056e1ebdeeb7d380c8da7bb130ba13f734069a5f385582" exitCode=0 Sep 10 17:40:25.729018 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:25.728914 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4tz2" event={"ID":"e3708abb-21c9-4590-90a9-00cfec61f382","Type":"ContainerDied","Data":"505ce2f542ba6dd6ca056e1ebdeeb7d380c8da7bb130ba13f734069a5f385582"} Sep 10 17:40:26.432485 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:26.432458 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:26.432946 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:26.432534 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:26.432946 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:26.432579 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:26.432946 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:26.432621 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:27.431689 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:27.431654 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:27.431830 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:27.431761 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:27.735665 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:27.735573 2565 generic.go:356] "Generic (PLEG): container finished" podID="e3708abb-21c9-4590-90a9-00cfec61f382" containerID="0490f79c86fb8974cbae99f29abbd9023ebe10a638adb60e969cb76ba1b18c19" exitCode=0 Sep 10 17:40:27.735665 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:27.735621 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4tz2" event={"ID":"e3708abb-21c9-4590-90a9-00cfec61f382","Type":"ContainerDied","Data":"0490f79c86fb8974cbae99f29abbd9023ebe10a638adb60e969cb76ba1b18c19"} Sep 10 17:40:28.432224 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:28.432191 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:28.432224 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:28.432208 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:28.432429 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:28.432327 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:28.432491 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:28.432466 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:29.197743 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:29.197709 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:29.198186 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:29.197755 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs\") pod \"network-metrics-daemon-cvx9n\" (UID: \"b563ccdb-296e-4358-81f7-82dc6bd66435\") " pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:29.198186 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:29.197857 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 17:40:29.198186 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:29.197930 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret podName:af672f7a-0702-4a9b-8a25-5cab8441f3c2 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:01.197912797 +0000 UTC m=+65.262323490 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret") pod "global-pull-secret-syncer-s42t8" (UID: "af672f7a-0702-4a9b-8a25-5cab8441f3c2") : object "kube-system"/"original-pull-secret" not registered Sep 10 17:40:29.198186 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:29.197968 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:40:29.198186 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:29.198028 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs podName:b563ccdb-296e-4358-81f7-82dc6bd66435 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:01.198010849 +0000 UTC m=+65.262421538 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs") pod "network-metrics-daemon-cvx9n" (UID: "b563ccdb-296e-4358-81f7-82dc6bd66435") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 17:40:29.298108 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:29.298077 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggcw7\" (UniqueName: \"kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7\") pod \"network-check-target-5k8rp\" (UID: \"c3d44764-8e31-4aaf-a532-86de76fc6820\") " pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:29.298242 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:29.298189 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 17:40:29.298242 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:29.298202 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 17:40:29.298242 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:29.298210 2565 projected.go:196] Error preparing data for projected volume kube-api-access-ggcw7 for pod openshift-network-diagnostics/network-check-target-5k8rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:40:29.298344 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:29.298252 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7 podName:c3d44764-8e31-4aaf-a532-86de76fc6820 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:01.29824151 +0000 UTC m=+65.362652199 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggcw7" (UniqueName: "kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7") pod "network-check-target-5k8rp" (UID: "c3d44764-8e31-4aaf-a532-86de76fc6820") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 17:40:29.431551 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:29.431524 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:29.431736 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:29.431647 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cvx9n" podUID="b563ccdb-296e-4358-81f7-82dc6bd66435" Sep 10 17:40:29.742352 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:29.742320 2565 generic.go:356] "Generic (PLEG): container finished" podID="e3708abb-21c9-4590-90a9-00cfec61f382" containerID="df51a447b37ab4725f01bdcc5e6d84114ddce08444a1638b2e109e766e943b8c" exitCode=0 Sep 10 17:40:29.742618 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:29.742360 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4tz2" event={"ID":"e3708abb-21c9-4590-90a9-00cfec61f382","Type":"ContainerDied","Data":"df51a447b37ab4725f01bdcc5e6d84114ddce08444a1638b2e109e766e943b8c"} Sep 10 17:40:30.432254 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:30.432217 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:30.432703 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:30.432350 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s42t8" podUID="af672f7a-0702-4a9b-8a25-5cab8441f3c2" Sep 10 17:40:30.432703 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:30.432405 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:30.432703 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:30.432520 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5k8rp" podUID="c3d44764-8e31-4aaf-a532-86de76fc6820" Sep 10 17:40:31.250029 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.249998 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-84.ec2.internal" event="NodeReady" Sep 10 17:40:31.250187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.250122 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 17:40:31.296982 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.296943 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gnq56"] Sep 10 17:40:31.320531 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.320505 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sx5vt"] Sep 10 17:40:31.320699 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.320635 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:31.322728 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.322694 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 17:40:31.322864 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.322745 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 17:40:31.323066 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.323034 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dwwmp\"" Sep 10 17:40:31.336468 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.336445 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gnq56"] Sep 10 17:40:31.336582 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.336483 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sx5vt"] Sep 10 17:40:31.336634 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.336582 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sx5vt" Sep 10 17:40:31.338827 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.338692 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 17:40:31.338827 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.338705 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 17:40:31.339051 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.338973 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7xd5n\"" Sep 10 17:40:31.339051 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.338989 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 17:40:31.418571 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.418538 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/25d7cbef-d614-40ec-bf7c-517c5e752cea-config-volume\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:31.418736 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.418591 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:31.418736 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.418606 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25d7cbef-d614-40ec-bf7c-517c5e752cea-tmp-dir\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:31.418736 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.418656 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97rqh\" (UniqueName: \"kubernetes.io/projected/25d7cbef-d614-40ec-bf7c-517c5e752cea-kube-api-access-97rqh\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:31.423840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.423807 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:40:31.431669 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.431651 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:40:31.433845 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.433810 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 17:40:31.434219 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.434025 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d22m9\"" Sep 10 17:40:31.518996 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.518928 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hjss\" (UniqueName: \"kubernetes.io/projected/5e2f1a90-af6b-49cc-9466-57b7848dcaf3-kube-api-access-2hjss\") pod \"ingress-canary-sx5vt\" (UID: \"5e2f1a90-af6b-49cc-9466-57b7848dcaf3\") " pod="openshift-ingress-canary/ingress-canary-sx5vt" Sep 10 17:40:31.518996 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.518964 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97rqh\" (UniqueName: \"kubernetes.io/projected/25d7cbef-d614-40ec-bf7c-517c5e752cea-kube-api-access-97rqh\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:31.519176 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.519043 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/25d7cbef-d614-40ec-bf7c-517c5e752cea-config-volume\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:31.519176 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.519096 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:31.519176 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.519112 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25d7cbef-d614-40ec-bf7c-517c5e752cea-tmp-dir\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:31.519176 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.519128 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e2f1a90-af6b-49cc-9466-57b7848dcaf3-cert\") pod \"ingress-canary-sx5vt\" (UID: \"5e2f1a90-af6b-49cc-9466-57b7848dcaf3\") " pod="openshift-ingress-canary/ingress-canary-sx5vt" Sep 10 17:40:31.519370 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:31.519332 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 17:40:31.519421 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:31.519385 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls podName:25d7cbef-d614-40ec-bf7c-517c5e752cea nodeName:}" failed. No retries permitted until 2026-09-10 17:40:32.019368087 +0000 UTC m=+36.083778781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls") pod "dns-default-gnq56" (UID: "25d7cbef-d614-40ec-bf7c-517c5e752cea") : secret "dns-default-metrics-tls" not found Sep 10 17:40:31.519513 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.519496 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25d7cbef-d614-40ec-bf7c-517c5e752cea-tmp-dir\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:31.519780 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.519763 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/25d7cbef-d614-40ec-bf7c-517c5e752cea-config-volume\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:31.530017 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.529854 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97rqh\" (UniqueName: \"kubernetes.io/projected/25d7cbef-d614-40ec-bf7c-517c5e752cea-kube-api-access-97rqh\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:31.619898 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.619849 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e2f1a90-af6b-49cc-9466-57b7848dcaf3-cert\") pod \"ingress-canary-sx5vt\" (UID: \"5e2f1a90-af6b-49cc-9466-57b7848dcaf3\") " pod="openshift-ingress-canary/ingress-canary-sx5vt" Sep 10 17:40:31.620031 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.619904 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hjss\" (UniqueName: \"kubernetes.io/projected/5e2f1a90-af6b-49cc-9466-57b7848dcaf3-kube-api-access-2hjss\") pod \"ingress-canary-sx5vt\" (UID: \"5e2f1a90-af6b-49cc-9466-57b7848dcaf3\") " pod="openshift-ingress-canary/ingress-canary-sx5vt" Sep 10 17:40:31.622160 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.622139 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e2f1a90-af6b-49cc-9466-57b7848dcaf3-cert\") pod \"ingress-canary-sx5vt\" (UID: \"5e2f1a90-af6b-49cc-9466-57b7848dcaf3\") " pod="openshift-ingress-canary/ingress-canary-sx5vt" Sep 10 17:40:31.627545 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.627523 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hjss\" (UniqueName: \"kubernetes.io/projected/5e2f1a90-af6b-49cc-9466-57b7848dcaf3-kube-api-access-2hjss\") pod \"ingress-canary-sx5vt\" (UID: \"5e2f1a90-af6b-49cc-9466-57b7848dcaf3\") " pod="openshift-ingress-canary/ingress-canary-sx5vt" Sep 10 17:40:31.645487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.645469 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sx5vt" Sep 10 17:40:31.967968 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:31.967932 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sx5vt"] Sep 10 17:40:31.972223 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:40:31.972188 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2f1a90_af6b_49cc_9466_57b7848dcaf3.slice/crio-aeb7af4c71ee888a70cd129463f5ce57e3344c22404823bfe5acd2ec0c4debbd WatchSource:0}: Error finding container aeb7af4c71ee888a70cd129463f5ce57e3344c22404823bfe5acd2ec0c4debbd: Status 404 returned error can't find the container with id aeb7af4c71ee888a70cd129463f5ce57e3344c22404823bfe5acd2ec0c4debbd Sep 10 17:40:32.023963 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:32.023931 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:32.024085 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:32.024066 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 17:40:32.024152 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:32.024140 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls podName:25d7cbef-d614-40ec-bf7c-517c5e752cea nodeName:}" failed. No retries permitted until 2026-09-10 17:40:33.024119648 +0000 UTC m=+37.088530339 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls") pod "dns-default-gnq56" (UID: "25d7cbef-d614-40ec-bf7c-517c5e752cea") : secret "dns-default-metrics-tls" not found Sep 10 17:40:32.399939 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:32.399904 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:40:32.431838 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:32.431814 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:40:32.432035 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:32.432015 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:40:32.435013 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:32.434993 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 17:40:32.435529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:32.435024 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gjm6q\"" Sep 10 17:40:32.435529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:32.434994 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 17:40:32.435529 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:32.434993 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 17:40:32.463054 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:32.463026 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:40:32.752376 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:32.752289 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sx5vt" event={"ID":"5e2f1a90-af6b-49cc-9466-57b7848dcaf3","Type":"ContainerStarted","Data":"aeb7af4c71ee888a70cd129463f5ce57e3344c22404823bfe5acd2ec0c4debbd"} Sep 10 17:40:33.032502 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:33.032406 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:33.032661 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:33.032612 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 17:40:33.032731 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:33.032720 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls podName:25d7cbef-d614-40ec-bf7c-517c5e752cea nodeName:}" failed. No retries permitted until 2026-09-10 17:40:35.032704097 +0000 UTC m=+39.097114800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls") pod "dns-default-gnq56" (UID: "25d7cbef-d614-40ec-bf7c-517c5e752cea") : secret "dns-default-metrics-tls" not found Sep 10 17:40:35.046048 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:35.046009 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:35.046569 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:35.046131 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 17:40:35.046569 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:35.046197 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls podName:25d7cbef-d614-40ec-bf7c-517c5e752cea nodeName:}" failed. No retries permitted until 2026-09-10 17:40:39.04618398 +0000 UTC m=+43.110594668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls") pod "dns-default-gnq56" (UID: "25d7cbef-d614-40ec-bf7c-517c5e752cea") : secret "dns-default-metrics-tls" not found Sep 10 17:40:35.900832 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:35.900802 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:40:35.970330 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:35.970290 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:40:36.428511 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:36.428442 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:40:36.498404 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:36.498373 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:40:36.763187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:36.763106 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sx5vt" event={"ID":"5e2f1a90-af6b-49cc-9466-57b7848dcaf3","Type":"ContainerStarted","Data":"acab799dfe4a9a12e16d192943c5b6d02abe1860a258d073a704cb9a997609d7"} Sep 10 17:40:36.765695 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:36.765667 2565 generic.go:356] "Generic (PLEG): container finished" podID="e3708abb-21c9-4590-90a9-00cfec61f382" containerID="e3196762a54672e667195e696b9dfa9ae80e73d882fd5fe090552ebfa08989dc" exitCode=0 Sep 10 17:40:36.765842 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:36.765706 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4tz2" event={"ID":"e3708abb-21c9-4590-90a9-00cfec61f382","Type":"ContainerDied","Data":"e3196762a54672e667195e696b9dfa9ae80e73d882fd5fe090552ebfa08989dc"} Sep 10 17:40:36.782929 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:36.781481 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sx5vt" podStartSLOduration=1.710834616 podStartE2EDuration="5.781462251s" totalImagesPullingTime="4.070627635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:36.779765094 +0000 UTC m=+40.844175805" watchObservedRunningTime="2026-09-10 17:40:36.781462251 +0000 UTC m=+40.845872964" Sep 10 17:40:37.770648 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:37.770615 2565 generic.go:356] "Generic (PLEG): container finished" podID="e3708abb-21c9-4590-90a9-00cfec61f382" containerID="33a0abbdcf575f4713345205c67c977fee0120643ef005bf3f4759953394d45a" exitCode=0 Sep 10 17:40:37.771049 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:37.770702 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4tz2" event={"ID":"e3708abb-21c9-4590-90a9-00cfec61f382","Type":"ContainerDied","Data":"33a0abbdcf575f4713345205c67c977fee0120643ef005bf3f4759953394d45a"} Sep 10 17:40:38.776746 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:38.776710 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4tz2" event={"ID":"e3708abb-21c9-4590-90a9-00cfec61f382","Type":"ContainerStarted","Data":"6245559dc25f86116cd30bc9e088a0683f1fec00c0edfb3238226290457002ee"} Sep 10 17:40:38.806397 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:38.806345 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-r4tz2" podStartSLOduration=7.9070128 podStartE2EDuration="41.806295684s" totalImagesPullingTime="33.899282884s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:39:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:38.805704961 +0000 UTC m=+42.870115671" watchObservedRunningTime="2026-09-10 17:40:38.806295684 +0000 UTC m=+42.870706394" Sep 10 17:40:38.929053 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:38.929019 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:40:38.990738 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:38.990707 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:40:39.076447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:39.076371 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:39.076575 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:39.076511 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 17:40:39.076632 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:39.076581 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls podName:25d7cbef-d614-40ec-bf7c-517c5e752cea nodeName:}" failed. No retries permitted until 2026-09-10 17:40:47.076560366 +0000 UTC m=+51.140971057 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls") pod "dns-default-gnq56" (UID: "25d7cbef-d614-40ec-bf7c-517c5e752cea") : secret "dns-default-metrics-tls" not found Sep 10 17:40:47.133268 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:47.133213 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:40:47.133738 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:47.133338 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 17:40:47.133738 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:40:47.133400 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls podName:25d7cbef-d614-40ec-bf7c-517c5e752cea nodeName:}" failed. No retries permitted until 2026-09-10 17:41:03.133384808 +0000 UTC m=+67.197795498 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls") pod "dns-default-gnq56" (UID: "25d7cbef-d614-40ec-bf7c-517c5e752cea") : secret "dns-default-metrics-tls" not found Sep 10 17:40:55.739299 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:55.739271 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g2pbs" Sep 10 17:40:55.881142 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:55.881089 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:40:57.049023 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:40:57.048992 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:01.227363 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.227306 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:41:01.227363 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.227370 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs\") pod \"network-metrics-daemon-cvx9n\" (UID: \"b563ccdb-296e-4358-81f7-82dc6bd66435\") " pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:41:01.229695 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.229675 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 17:41:01.229763 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.229682 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 17:41:01.237934 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:01.237913 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 17:41:01.238040 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:01.237966 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs podName:b563ccdb-296e-4358-81f7-82dc6bd66435 nodeName:}" failed. No retries permitted until 2026-09-10 17:42:05.237951835 +0000 UTC m=+129.302362523 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs") pod "network-metrics-daemon-cvx9n" (UID: "b563ccdb-296e-4358-81f7-82dc6bd66435") : secret "metrics-daemon-secret" not found Sep 10 17:41:01.240197 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.240176 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af672f7a-0702-4a9b-8a25-5cab8441f3c2-original-pull-secret\") pod \"global-pull-secret-syncer-s42t8\" (UID: \"af672f7a-0702-4a9b-8a25-5cab8441f3c2\") " pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:41:01.245147 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.245129 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s42t8" Sep 10 17:41:01.327865 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.327835 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggcw7\" (UniqueName: \"kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7\") pod \"network-check-target-5k8rp\" (UID: \"c3d44764-8e31-4aaf-a532-86de76fc6820\") " pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:41:01.330373 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.330353 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 17:41:01.340299 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.340281 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 17:41:01.351210 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.351186 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggcw7\" (UniqueName: \"kubernetes.io/projected/c3d44764-8e31-4aaf-a532-86de76fc6820-kube-api-access-ggcw7\") pod \"network-check-target-5k8rp\" (UID: \"c3d44764-8e31-4aaf-a532-86de76fc6820\") " pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:41:01.510536 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.510340 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s42t8"] Sep 10 17:41:01.512860 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:41:01.512832 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf672f7a_0702_4a9b_8a25_5cab8441f3c2.slice/crio-bcc4cdad5c6b4adb656725fe8d65340ead47976e2b67858920b3898240f70ebe WatchSource:0}: Error finding container bcc4cdad5c6b4adb656725fe8d65340ead47976e2b67858920b3898240f70ebe: Status 404 returned error can't find the container with id bcc4cdad5c6b4adb656725fe8d65340ead47976e2b67858920b3898240f70ebe Sep 10 17:41:01.552207 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.552183 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gjm6q\"" Sep 10 17:41:01.561039 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.561020 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:41:01.823046 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.822978 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5k8rp"] Sep 10 17:41:01.826038 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:41:01.826013 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d44764_8e31_4aaf_a532_86de76fc6820.slice/crio-8da0470723918973a0add6eb585ad2960b0a1d11db5b521e271a8bee1bb0d9b8 WatchSource:0}: Error finding container 8da0470723918973a0add6eb585ad2960b0a1d11db5b521e271a8bee1bb0d9b8: Status 404 returned error can't find the container with id 8da0470723918973a0add6eb585ad2960b0a1d11db5b521e271a8bee1bb0d9b8 Sep 10 17:41:01.830847 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.830819 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5k8rp" event={"ID":"c3d44764-8e31-4aaf-a532-86de76fc6820","Type":"ContainerStarted","Data":"8da0470723918973a0add6eb585ad2960b0a1d11db5b521e271a8bee1bb0d9b8"} Sep 10 17:41:01.831918 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:01.831875 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s42t8" event={"ID":"af672f7a-0702-4a9b-8a25-5cab8441f3c2","Type":"ContainerStarted","Data":"bcc4cdad5c6b4adb656725fe8d65340ead47976e2b67858920b3898240f70ebe"} Sep 10 17:41:03.140455 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:03.140415 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:41:03.140924 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:03.140558 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 17:41:03.140924 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:03.140628 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls podName:25d7cbef-d614-40ec-bf7c-517c5e752cea nodeName:}" failed. No retries permitted until 2026-09-10 17:41:35.140608042 +0000 UTC m=+99.205018746 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls") pod "dns-default-gnq56" (UID: "25d7cbef-d614-40ec-bf7c-517c5e752cea") : secret "dns-default-metrics-tls" not found Sep 10 17:41:06.845852 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:06.845812 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5k8rp" event={"ID":"c3d44764-8e31-4aaf-a532-86de76fc6820","Type":"ContainerStarted","Data":"88d0fb42eb5252347d1558ade87a52aa928659f56244f062e722a88dd0446a94"} Sep 10 17:41:06.846294 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:06.845952 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:41:06.847264 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:06.847242 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s42t8" event={"ID":"af672f7a-0702-4a9b-8a25-5cab8441f3c2","Type":"ContainerStarted","Data":"4eb1055db7dd97ad4c13f56c71a8e2f9c9dfbc616ba540637f8d1f65b12a2f0d"} Sep 10 17:41:06.862411 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:06.862372 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5k8rp" podStartSLOduration=65.453035408 podStartE2EDuration="1m9.862357615s" totalImagesPullingTime="4.409322207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:06.861306279 +0000 UTC m=+70.925716989" watchObservedRunningTime="2026-09-10 17:41:06.862357615 +0000 UTC m=+70.926768325" Sep 10 17:41:06.883260 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:06.883223 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-s42t8" podStartSLOduration=65.155559594 podStartE2EDuration="1m9.883210383s" totalImagesPullingTime="4.727650789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:06.882371118 +0000 UTC m=+70.946781828" watchObservedRunningTime="2026-09-10 17:41:06.883210383 +0000 UTC m=+70.947621094" Sep 10 17:41:21.393048 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.392931 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb"] Sep 10 17:41:21.395269 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.395251 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:21.402005 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.399637 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 17:41:21.402005 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.400384 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 10 17:41:21.402005 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.400644 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nftrz\"" Sep 10 17:41:21.402005 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.401223 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 17:41:21.402005 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.401402 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 10 17:41:21.412532 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.412500 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb"] Sep 10 17:41:21.468197 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.468159 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:21.468378 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.468212 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxkdn\" (UniqueName: \"kubernetes.io/projected/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-kube-api-access-sxkdn\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:21.468378 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.468343 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:21.501894 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.501847 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s"] Sep 10 17:41:21.503485 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.503469 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" Sep 10 17:41:21.507091 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.507052 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 10 17:41:21.507302 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.507166 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 10 17:41:21.507371 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.507358 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 10 17:41:21.507498 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.507482 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 10 17:41:21.508168 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.508152 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-98bkm\"" Sep 10 17:41:21.516855 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.516830 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s"] Sep 10 17:41:21.568803 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.568764 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:21.569021 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.568826 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxkdn\" (UniqueName: \"kubernetes.io/projected/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-kube-api-access-sxkdn\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:21.569021 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.568858 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/06559b57-0c49-4988-87ab-01fde08339dd-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7tr6s\" (UID: \"06559b57-0c49-4988-87ab-01fde08339dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" Sep 10 17:41:21.569021 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:21.568928 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:21.569021 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:21.568998 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls podName:e81e5b8d-0807-48b8-8bf7-d31d799e2d40 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:22.068982905 +0000 UTC m=+86.133393600 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-fx8qb" (UID: "e81e5b8d-0807-48b8-8bf7-d31d799e2d40") : secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:21.569021 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.568938 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:21.569299 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.569063 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/06559b57-0c49-4988-87ab-01fde08339dd-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7tr6s\" (UID: \"06559b57-0c49-4988-87ab-01fde08339dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" Sep 10 17:41:21.569299 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.569095 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn4p6\" (UniqueName: \"kubernetes.io/projected/06559b57-0c49-4988-87ab-01fde08339dd-kube-api-access-zn4p6\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7tr6s\" (UID: \"06559b57-0c49-4988-87ab-01fde08339dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" Sep 10 17:41:21.569668 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.569644 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:21.577415 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.577389 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxkdn\" (UniqueName: \"kubernetes.io/projected/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-kube-api-access-sxkdn\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:21.597785 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.597749 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq"] Sep 10 17:41:21.599827 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.599806 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" Sep 10 17:41:21.602008 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.601980 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 10 17:41:21.602117 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.602015 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 10 17:41:21.602117 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.602076 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 10 17:41:21.602274 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.602261 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-j94qh\"" Sep 10 17:41:21.608940 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.608915 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq"] Sep 10 17:41:21.670159 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.670070 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-twqmq\" (UID: \"cc2f0d42-0cdf-479e-afbd-fdbfa1672991\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" Sep 10 17:41:21.670159 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.670106 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/06559b57-0c49-4988-87ab-01fde08339dd-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7tr6s\" (UID: \"06559b57-0c49-4988-87ab-01fde08339dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" Sep 10 17:41:21.670159 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.670149 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/06559b57-0c49-4988-87ab-01fde08339dd-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7tr6s\" (UID: \"06559b57-0c49-4988-87ab-01fde08339dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" Sep 10 17:41:21.670389 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.670190 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn4p6\" (UniqueName: \"kubernetes.io/projected/06559b57-0c49-4988-87ab-01fde08339dd-kube-api-access-zn4p6\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7tr6s\" (UID: \"06559b57-0c49-4988-87ab-01fde08339dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" Sep 10 17:41:21.670389 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.670241 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skd9b\" (UniqueName: \"kubernetes.io/projected/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-kube-api-access-skd9b\") pod \"cluster-samples-operator-7cdcfcf76c-twqmq\" (UID: \"cc2f0d42-0cdf-479e-afbd-fdbfa1672991\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" Sep 10 17:41:21.670729 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.670708 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/06559b57-0c49-4988-87ab-01fde08339dd-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7tr6s\" (UID: \"06559b57-0c49-4988-87ab-01fde08339dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" Sep 10 17:41:21.672385 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.672362 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/06559b57-0c49-4988-87ab-01fde08339dd-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7tr6s\" (UID: \"06559b57-0c49-4988-87ab-01fde08339dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" Sep 10 17:41:21.678378 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.678354 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn4p6\" (UniqueName: \"kubernetes.io/projected/06559b57-0c49-4988-87ab-01fde08339dd-kube-api-access-zn4p6\") pod \"kube-storage-version-migrator-operator-8d7dd4675-7tr6s\" (UID: \"06559b57-0c49-4988-87ab-01fde08339dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" Sep 10 17:41:21.704247 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.704222 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b69d56847-wcx2s"] Sep 10 17:41:21.706144 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.706130 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.707537 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.707516 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:21.709209 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.709188 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 17:41:21.709327 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.709170 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 17:41:21.709410 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.709238 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 17:41:21.709618 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.709603 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g6c28\"" Sep 10 17:41:21.714755 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.714738 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 17:41:21.722086 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.722067 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b69d56847-wcx2s"] Sep 10 17:41:21.771012 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.770980 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skd9b\" (UniqueName: \"kubernetes.io/projected/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-kube-api-access-skd9b\") pod \"cluster-samples-operator-7cdcfcf76c-twqmq\" (UID: \"cc2f0d42-0cdf-479e-afbd-fdbfa1672991\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" Sep 10 17:41:21.771190 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.771017 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1683c301-dec1-47bd-b9d8-39d509013d6d-image-registry-private-configuration\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.771190 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.771035 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr6m9\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-kube-api-access-pr6m9\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.771190 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.771072 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.771190 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.771140 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-certificates\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.771397 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.771193 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-bound-sa-token\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.771397 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.771214 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1683c301-dec1-47bd-b9d8-39d509013d6d-ca-trust-extracted\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.771397 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.771263 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-twqmq\" (UID: \"cc2f0d42-0cdf-479e-afbd-fdbfa1672991\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" Sep 10 17:41:21.771397 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.771292 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1683c301-dec1-47bd-b9d8-39d509013d6d-trusted-ca\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.771397 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.771330 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1683c301-dec1-47bd-b9d8-39d509013d6d-installation-pull-secrets\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.771637 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:21.771399 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 17:41:21.771637 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:21.771475 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls podName:cc2f0d42-0cdf-479e-afbd-fdbfa1672991 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:22.271455233 +0000 UTC m=+86.335865924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-twqmq" (UID: "cc2f0d42-0cdf-479e-afbd-fdbfa1672991") : secret "samples-operator-tls" not found Sep 10 17:41:21.785613 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.785582 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skd9b\" (UniqueName: \"kubernetes.io/projected/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-kube-api-access-skd9b\") pod \"cluster-samples-operator-7cdcfcf76c-twqmq\" (UID: \"cc2f0d42-0cdf-479e-afbd-fdbfa1672991\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" Sep 10 17:41:21.791438 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.791418 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:21.812673 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.812646 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" Sep 10 17:41:21.871807 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.871775 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.871959 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.871815 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-certificates\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.871959 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:21.871943 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 17:41:21.872072 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:21.871961 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b69d56847-wcx2s: secret "image-registry-tls" not found Sep 10 17:41:21.872072 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:21.872022 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls podName:1683c301-dec1-47bd-b9d8-39d509013d6d nodeName:}" failed. No retries permitted until 2026-09-10 17:41:22.372003535 +0000 UTC m=+86.436414242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls") pod "image-registry-7b69d56847-wcx2s" (UID: "1683c301-dec1-47bd-b9d8-39d509013d6d") : secret "image-registry-tls" not found Sep 10 17:41:21.872319 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.872217 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-bound-sa-token\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.872319 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.872262 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1683c301-dec1-47bd-b9d8-39d509013d6d-ca-trust-extracted\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.872443 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.872320 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1683c301-dec1-47bd-b9d8-39d509013d6d-trusted-ca\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.872443 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.872387 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1683c301-dec1-47bd-b9d8-39d509013d6d-installation-pull-secrets\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.872550 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.872448 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1683c301-dec1-47bd-b9d8-39d509013d6d-image-registry-private-configuration\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.872550 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.872477 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr6m9\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-kube-api-access-pr6m9\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.873784 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.872822 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-certificates\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.873784 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.873533 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1683c301-dec1-47bd-b9d8-39d509013d6d-ca-trust-extracted\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.873784 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.873731 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1683c301-dec1-47bd-b9d8-39d509013d6d-trusted-ca\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.875432 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.875412 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1683c301-dec1-47bd-b9d8-39d509013d6d-image-registry-private-configuration\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.875522 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.875482 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1683c301-dec1-47bd-b9d8-39d509013d6d-installation-pull-secrets\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.883713 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.883692 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr6m9\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-kube-api-access-pr6m9\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:21.887174 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:21.887132 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-bound-sa-token\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:22.074090 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:22.074014 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:22.074223 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:22.074192 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:22.074277 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:22.074266 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls podName:e81e5b8d-0807-48b8-8bf7-d31d799e2d40 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:23.074249868 +0000 UTC m=+87.138660557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-fx8qb" (UID: "e81e5b8d-0807-48b8-8bf7-d31d799e2d40") : secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:22.078607 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:22.078580 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s"] Sep 10 17:41:22.081200 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:41:22.081175 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06559b57_0c49_4988_87ab_01fde08339dd.slice/crio-20b410414ce802b3c7e8dce37efced12a7e04b32331eda34ae2ff7565159fc82 WatchSource:0}: Error finding container 20b410414ce802b3c7e8dce37efced12a7e04b32331eda34ae2ff7565159fc82: Status 404 returned error can't find the container with id 20b410414ce802b3c7e8dce37efced12a7e04b32331eda34ae2ff7565159fc82 Sep 10 17:41:22.276439 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:22.276405 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-twqmq\" (UID: \"cc2f0d42-0cdf-479e-afbd-fdbfa1672991\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" Sep 10 17:41:22.276656 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:22.276577 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 17:41:22.276725 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:22.276664 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls podName:cc2f0d42-0cdf-479e-afbd-fdbfa1672991 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:23.276642624 +0000 UTC m=+87.341053314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-twqmq" (UID: "cc2f0d42-0cdf-479e-afbd-fdbfa1672991") : secret "samples-operator-tls" not found Sep 10 17:41:22.376856 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:22.376744 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:22.377043 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:22.376908 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 17:41:22.377043 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:22.376928 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b69d56847-wcx2s: secret "image-registry-tls" not found Sep 10 17:41:22.377043 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:22.376980 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls podName:1683c301-dec1-47bd-b9d8-39d509013d6d nodeName:}" failed. No retries permitted until 2026-09-10 17:41:23.376965968 +0000 UTC m=+87.441376657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls") pod "image-registry-7b69d56847-wcx2s" (UID: "1683c301-dec1-47bd-b9d8-39d509013d6d") : secret "image-registry-tls" not found Sep 10 17:41:22.888268 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:22.888236 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" event={"ID":"06559b57-0c49-4988-87ab-01fde08339dd","Type":"ContainerStarted","Data":"20b410414ce802b3c7e8dce37efced12a7e04b32331eda34ae2ff7565159fc82"} Sep 10 17:41:23.082260 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:23.082213 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:23.082444 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:23.082364 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:23.082444 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:23.082432 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls podName:e81e5b8d-0807-48b8-8bf7-d31d799e2d40 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:25.082416914 +0000 UTC m=+89.146827603 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-fx8qb" (UID: "e81e5b8d-0807-48b8-8bf7-d31d799e2d40") : secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:23.284201 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:23.284099 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-twqmq\" (UID: \"cc2f0d42-0cdf-479e-afbd-fdbfa1672991\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" Sep 10 17:41:23.284363 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:23.284244 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 17:41:23.284363 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:23.284306 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls podName:cc2f0d42-0cdf-479e-afbd-fdbfa1672991 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:25.284291359 +0000 UTC m=+89.348702064 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-twqmq" (UID: "cc2f0d42-0cdf-479e-afbd-fdbfa1672991") : secret "samples-operator-tls" not found Sep 10 17:41:23.385258 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:23.385214 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:23.385427 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:23.385361 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 17:41:23.385427 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:23.385383 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b69d56847-wcx2s: secret "image-registry-tls" not found Sep 10 17:41:23.385505 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:23.385438 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls podName:1683c301-dec1-47bd-b9d8-39d509013d6d nodeName:}" failed. No retries permitted until 2026-09-10 17:41:25.38542275 +0000 UTC m=+89.449833439 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls") pod "image-registry-7b69d56847-wcx2s" (UID: "1683c301-dec1-47bd-b9d8-39d509013d6d") : secret "image-registry-tls" not found Sep 10 17:41:23.620445 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:23.620328 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:23.692548 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:23.692516 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:24.895157 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:24.895127 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" event={"ID":"06559b57-0c49-4988-87ab-01fde08339dd","Type":"ContainerStarted","Data":"939512c3b3435fe06cd698b92491a065237f3bd0d57b2ce19c63ce5a71e922f0"} Sep 10 17:41:24.911353 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:24.911306 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" podStartSLOduration=1.719385035 podStartE2EDuration="3.911289423s" totalImagesPullingTime="2.191904388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:24.91011629 +0000 UTC m=+88.974527020" watchObservedRunningTime="2026-09-10 17:41:24.911289423 +0000 UTC m=+88.975700133" Sep 10 17:41:25.098242 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.098194 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:25.098421 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:25.098338 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:25.098421 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:25.098407 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls podName:e81e5b8d-0807-48b8-8bf7-d31d799e2d40 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:29.098390725 +0000 UTC m=+93.162801427 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-fx8qb" (UID: "e81e5b8d-0807-48b8-8bf7-d31d799e2d40") : secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:25.299639 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.299534 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-twqmq\" (UID: \"cc2f0d42-0cdf-479e-afbd-fdbfa1672991\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" Sep 10 17:41:25.299805 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:25.299684 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 17:41:25.299805 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:25.299754 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls podName:cc2f0d42-0cdf-479e-afbd-fdbfa1672991 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:29.299736254 +0000 UTC m=+93.364146956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-twqmq" (UID: "cc2f0d42-0cdf-479e-afbd-fdbfa1672991") : secret "samples-operator-tls" not found Sep 10 17:41:25.400753 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.400697 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:25.400959 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:25.400851 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 17:41:25.400959 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:25.400873 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b69d56847-wcx2s: secret "image-registry-tls" not found Sep 10 17:41:25.400959 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:25.400944 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls podName:1683c301-dec1-47bd-b9d8-39d509013d6d nodeName:}" failed. No retries permitted until 2026-09-10 17:41:29.400928263 +0000 UTC m=+93.465338952 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls") pod "image-registry-7b69d56847-wcx2s" (UID: "1683c301-dec1-47bd-b9d8-39d509013d6d") : secret "image-registry-tls" not found Sep 10 17:41:25.509111 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.509078 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:25.557651 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.557567 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-pkfvp"] Sep 10 17:41:25.559627 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.559598 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-pkfvp" Sep 10 17:41:25.562417 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.562389 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kh9hx\"" Sep 10 17:41:25.562542 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.562403 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 10 17:41:25.563438 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.563424 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 10 17:41:25.572760 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.572738 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-pkfvp"] Sep 10 17:41:25.602180 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.602145 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz55r\" (UniqueName: \"kubernetes.io/projected/8a8605cf-a1c3-4608-a2e5-e26aad1444d5-kube-api-access-dz55r\") pod \"migrator-fdbb8fdff-pkfvp\" (UID: \"8a8605cf-a1c3-4608-a2e5-e26aad1444d5\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-pkfvp" Sep 10 17:41:25.609572 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.609541 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:25.703562 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.703530 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz55r\" (UniqueName: \"kubernetes.io/projected/8a8605cf-a1c3-4608-a2e5-e26aad1444d5-kube-api-access-dz55r\") pod \"migrator-fdbb8fdff-pkfvp\" (UID: \"8a8605cf-a1c3-4608-a2e5-e26aad1444d5\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-pkfvp" Sep 10 17:41:25.706136 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.706101 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:25.715423 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.715399 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz55r\" (UniqueName: \"kubernetes.io/projected/8a8605cf-a1c3-4608-a2e5-e26aad1444d5-kube-api-access-dz55r\") pod \"migrator-fdbb8fdff-pkfvp\" (UID: \"8a8605cf-a1c3-4608-a2e5-e26aad1444d5\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-pkfvp" Sep 10 17:41:25.868466 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:25.868371 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-pkfvp" Sep 10 17:41:26.134943 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.134912 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-pkfvp"] Sep 10 17:41:26.138059 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:41:26.138036 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a8605cf_a1c3_4608_a2e5_e26aad1444d5.slice/crio-783bc04c6fdc5ab2f2a60c8a0b53c957e0f5137a428f4127474c3624552c9a4a WatchSource:0}: Error finding container 783bc04c6fdc5ab2f2a60c8a0b53c957e0f5137a428f4127474c3624552c9a4a: Status 404 returned error can't find the container with id 783bc04c6fdc5ab2f2a60c8a0b53c957e0f5137a428f4127474c3624552c9a4a Sep 10 17:41:26.350549 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.350519 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-w6tmf"] Sep 10 17:41:26.353144 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.353129 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:26.355389 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.355367 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 17:41:26.355389 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.355379 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 17:41:26.355963 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.355943 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 17:41:26.356043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.355967 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 17:41:26.356283 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.356261 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ktmw7\"" Sep 10 17:41:26.357516 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.357471 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:26.368267 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.368246 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w6tmf"] Sep 10 17:41:26.407215 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.407137 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:26.407215 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.407186 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlr2x\" (UniqueName: \"kubernetes.io/projected/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-api-access-rlr2x\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:26.407376 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.407239 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:26.407376 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.407275 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/326a773c-08de-4f4e-af1e-7c46b8b929c9-data-volume\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:26.407376 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.407327 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/326a773c-08de-4f4e-af1e-7c46b8b929c9-crio-socket\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:26.435035 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.434997 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:26.508417 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.508375 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlr2x\" (UniqueName: \"kubernetes.io/projected/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-api-access-rlr2x\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:26.508607 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.508434 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:26.508607 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.508463 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/326a773c-08de-4f4e-af1e-7c46b8b929c9-data-volume\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:26.508607 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.508506 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/326a773c-08de-4f4e-af1e-7c46b8b929c9-crio-socket\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:26.508607 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:26.508566 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 17:41:26.508808 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.508621 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/326a773c-08de-4f4e-af1e-7c46b8b929c9-crio-socket\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:26.508808 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:26.508644 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm podName:326a773c-08de-4f4e-af1e-7c46b8b929c9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:27.008624915 +0000 UTC m=+91.073035607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w6tmf" (UID: "326a773c-08de-4f4e-af1e-7c46b8b929c9") : configmap "kube-rbac-proxy" not found Sep 10 17:41:26.508808 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.508663 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:26.508808 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:26.508761 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 17:41:26.508808 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:26.508801 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls podName:326a773c-08de-4f4e-af1e-7c46b8b929c9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:27.008791359 +0000 UTC m=+91.073202052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w6tmf" (UID: "326a773c-08de-4f4e-af1e-7c46b8b929c9") : secret "insights-runtime-extractor-tls" not found Sep 10 17:41:26.508808 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.508803 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/326a773c-08de-4f4e-af1e-7c46b8b929c9-data-volume\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:26.516157 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.516134 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlr2x\" (UniqueName: \"kubernetes.io/projected/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-api-access-rlr2x\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:26.630547 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.630511 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:26.731684 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.731582 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:26.902415 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:26.902379 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-pkfvp" event={"ID":"8a8605cf-a1c3-4608-a2e5-e26aad1444d5","Type":"ContainerStarted","Data":"783bc04c6fdc5ab2f2a60c8a0b53c957e0f5137a428f4127474c3624552c9a4a"} Sep 10 17:41:27.012659 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:27.012568 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:27.012659 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:27.012631 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:27.012831 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:27.012706 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 17:41:27.012831 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:27.012728 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 17:41:27.012831 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:27.012793 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm podName:326a773c-08de-4f4e-af1e-7c46b8b929c9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:28.012772219 +0000 UTC m=+92.077182912 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w6tmf" (UID: "326a773c-08de-4f4e-af1e-7c46b8b929c9") : configmap "kube-rbac-proxy" not found Sep 10 17:41:27.012831 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:27.012810 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls podName:326a773c-08de-4f4e-af1e-7c46b8b929c9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:28.0128024 +0000 UTC m=+92.077213092 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w6tmf" (UID: "326a773c-08de-4f4e-af1e-7c46b8b929c9") : secret "insights-runtime-extractor-tls" not found Sep 10 17:41:27.059348 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:27.059312 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:27.122467 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:27.122434 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:27.395112 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:27.395082 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:27.495214 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:27.495191 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:27.907173 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:27.907138 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-pkfvp" event={"ID":"8a8605cf-a1c3-4608-a2e5-e26aad1444d5","Type":"ContainerStarted","Data":"ba9a8f6e97458112e7e2474cb29be2854737754c606e0bd66ecb090d05e69d83"} Sep 10 17:41:27.907335 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:27.907220 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-pkfvp" event={"ID":"8a8605cf-a1c3-4608-a2e5-e26aad1444d5","Type":"ContainerStarted","Data":"73cc8c8318d360e320b6ae0a71a9ee66942714aa535a6ba9d48123716c08bb23"} Sep 10 17:41:27.923552 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:27.923501 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-pkfvp" podStartSLOduration=1.605409682 podStartE2EDuration="2.923484395s" totalImagesPullingTime="1.318074713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:27.923015121 +0000 UTC m=+91.987425832" watchObservedRunningTime="2026-09-10 17:41:27.923484395 +0000 UTC m=+91.987895143" Sep 10 17:41:28.005789 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:28.005751 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:28.022518 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:28.022483 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:28.022692 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:28.022558 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:28.022692 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:28.022673 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 17:41:28.022794 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:28.022732 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls podName:326a773c-08de-4f4e-af1e-7c46b8b929c9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:30.022716254 +0000 UTC m=+94.087126943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w6tmf" (UID: "326a773c-08de-4f4e-af1e-7c46b8b929c9") : secret "insights-runtime-extractor-tls" not found Sep 10 17:41:28.022794 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:28.022675 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 17:41:28.022917 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:28.022832 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm podName:326a773c-08de-4f4e-af1e-7c46b8b929c9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:30.022813257 +0000 UTC m=+94.087223970 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w6tmf" (UID: "326a773c-08de-4f4e-af1e-7c46b8b929c9") : configmap "kube-rbac-proxy" not found Sep 10 17:41:28.106042 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:28.106004 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:28.974154 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:28.974119 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:29.130539 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:29.130495 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:29.130713 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:29.130637 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:29.130713 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:29.130694 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls podName:e81e5b8d-0807-48b8-8bf7-d31d799e2d40 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:37.130678837 +0000 UTC m=+101.195089527 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-fx8qb" (UID: "e81e5b8d-0807-48b8-8bf7-d31d799e2d40") : secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:29.290861 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:29.290787 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w2k5w_0c6a2e31-247f-4534-845e-8e68feecb206/dns-node-resolver/0.log" Sep 10 17:41:29.332780 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:29.332750 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-twqmq\" (UID: \"cc2f0d42-0cdf-479e-afbd-fdbfa1672991\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" Sep 10 17:41:29.332941 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:29.332922 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 17:41:29.333000 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:29.332990 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls podName:cc2f0d42-0cdf-479e-afbd-fdbfa1672991 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:37.332976117 +0000 UTC m=+101.397386806 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-twqmq" (UID: "cc2f0d42-0cdf-479e-afbd-fdbfa1672991") : secret "samples-operator-tls" not found Sep 10 17:41:29.433782 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:29.433740 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:29.433944 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:29.433880 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 17:41:29.433944 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:29.433913 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b69d56847-wcx2s: secret "image-registry-tls" not found Sep 10 17:41:29.434013 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:29.433963 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls podName:1683c301-dec1-47bd-b9d8-39d509013d6d nodeName:}" failed. No retries permitted until 2026-09-10 17:41:37.433948983 +0000 UTC m=+101.498359702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls") pod "image-registry-7b69d56847-wcx2s" (UID: "1683c301-dec1-47bd-b9d8-39d509013d6d") : secret "image-registry-tls" not found Sep 10 17:41:29.891271 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:29.891244 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-66qw8_dc0c8edc-89e0-4448-896c-78aa56a57d1c/node-ca/0.log" Sep 10 17:41:29.905471 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:29.905440 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:30.038973 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:30.038933 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:30.039345 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:30.039016 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:30.039345 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:30.039074 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 17:41:30.039345 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:30.039112 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 17:41:30.039345 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:30.039137 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm podName:326a773c-08de-4f4e-af1e-7c46b8b929c9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:34.039121751 +0000 UTC m=+98.103532440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w6tmf" (UID: "326a773c-08de-4f4e-af1e-7c46b8b929c9") : configmap "kube-rbac-proxy" not found Sep 10 17:41:30.039345 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:30.039153 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls podName:326a773c-08de-4f4e-af1e-7c46b8b929c9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:34.039145457 +0000 UTC m=+98.103556146 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w6tmf" (UID: "326a773c-08de-4f4e-af1e-7c46b8b929c9") : secret "insights-runtime-extractor-tls" not found Sep 10 17:41:31.091013 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:31.090980 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sx5vt_5e2f1a90-af6b-49cc-9466-57b7848dcaf3/serve-healthcheck-canary/0.log" Sep 10 17:41:31.291801 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:31.291776 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-pkfvp_8a8605cf-a1c3-4608-a2e5-e26aad1444d5/migrator/0.log" Sep 10 17:41:31.491278 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:31.491251 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-pkfvp_8a8605cf-a1c3-4608-a2e5-e26aad1444d5/graceful-termination/0.log" Sep 10 17:41:31.692005 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:31.691972 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-7tr6s_06559b57-0c49-4988-87ab-01fde08339dd/kube-storage-version-migrator-operator/0.log" Sep 10 17:41:32.861474 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:32.861442 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:33.037035 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:33.037002 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:33.135315 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:33.135287 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:33.236539 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:33.236508 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:33.355113 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:33.355069 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:33.766606 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:33.766565 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:34.073060 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:34.072956 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:34.073060 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:34.073050 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:34.073575 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:34.073100 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 17:41:34.073575 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:34.073161 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 17:41:34.073575 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:34.073181 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm podName:326a773c-08de-4f4e-af1e-7c46b8b929c9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:42.073158405 +0000 UTC m=+106.137569094 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w6tmf" (UID: "326a773c-08de-4f4e-af1e-7c46b8b929c9") : configmap "kube-rbac-proxy" not found Sep 10 17:41:34.073575 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:34.073202 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls podName:326a773c-08de-4f4e-af1e-7c46b8b929c9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:42.073190828 +0000 UTC m=+106.137601517 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w6tmf" (UID: "326a773c-08de-4f4e-af1e-7c46b8b929c9") : secret "insights-runtime-extractor-tls" not found Sep 10 17:41:34.364126 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:34.364038 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:34.960398 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:34.960354 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:35.158643 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:35.158616 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:35.183828 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:35.183795 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:41:35.183988 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:35.183956 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 17:41:35.184033 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:35.184013 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls podName:25d7cbef-d614-40ec-bf7c-517c5e752cea nodeName:}" failed. No retries permitted until 2026-09-10 17:42:39.183999063 +0000 UTC m=+163.248409752 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls") pod "dns-default-gnq56" (UID: "25d7cbef-d614-40ec-bf7c-517c5e752cea") : secret "dns-default-metrics-tls" not found Sep 10 17:41:35.360576 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:35.360468 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:35.557620 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:35.557585 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:36.455542 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:36.455510 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:36.556395 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:36.556348 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:36.713774 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:36.713677 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:36.792797 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:36.792762 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:36.957526 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:36.957493 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:37.161595 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.161554 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:37.200251 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.200209 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:37.200432 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:37.200355 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:37.200432 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:37.200420 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls podName:e81e5b8d-0807-48b8-8bf7-d31d799e2d40 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:53.200403932 +0000 UTC m=+117.264814620 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-fx8qb" (UID: "e81e5b8d-0807-48b8-8bf7-d31d799e2d40") : secret "cluster-monitoring-operator-tls" not found Sep 10 17:41:37.380935 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.380881 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:37.402082 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.402042 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-twqmq\" (UID: \"cc2f0d42-0cdf-479e-afbd-fdbfa1672991\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" Sep 10 17:41:37.404526 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.404505 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc2f0d42-0cdf-479e-afbd-fdbfa1672991-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-twqmq\" (UID: \"cc2f0d42-0cdf-479e-afbd-fdbfa1672991\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" Sep 10 17:41:37.482125 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.482038 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:37.503157 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.503123 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:37.505515 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.505496 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls\") pod \"image-registry-7b69d56847-wcx2s\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:37.509386 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.509361 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" Sep 10 17:41:37.586794 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.586758 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:37.615567 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.615539 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:37.818269 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.818247 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq"] Sep 10 17:41:37.851841 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.851811 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5k8rp" Sep 10 17:41:37.912164 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.912138 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b69d56847-wcx2s"] Sep 10 17:41:37.914974 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:41:37.914949 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1683c301_dec1_47bd_b9d8_39d509013d6d.slice/crio-b9a8d2d553b260b376e77051fa4305f3ed91c73e35642c4cdbc181e6dabd0a02 WatchSource:0}: Error finding container b9a8d2d553b260b376e77051fa4305f3ed91c73e35642c4cdbc181e6dabd0a02: Status 404 returned error can't find the container with id b9a8d2d553b260b376e77051fa4305f3ed91c73e35642c4cdbc181e6dabd0a02 Sep 10 17:41:37.938094 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.938052 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" event={"ID":"1683c301-dec1-47bd-b9d8-39d509013d6d","Type":"ContainerStarted","Data":"b9a8d2d553b260b376e77051fa4305f3ed91c73e35642c4cdbc181e6dabd0a02"} Sep 10 17:41:37.939167 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:37.939139 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" event={"ID":"cc2f0d42-0cdf-479e-afbd-fdbfa1672991","Type":"ContainerStarted","Data":"48bd1b9f074c03ef9aa6328272976e78deea520ab9c2936fd93c5a3a8afde3fb"} Sep 10 17:41:38.724475 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:38.724442 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:38.825125 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:38.825091 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:38.944113 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:38.944077 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" event={"ID":"1683c301-dec1-47bd-b9d8-39d509013d6d","Type":"ContainerStarted","Data":"10b1e5a223d14d0d1538a57f2c7c7e1c2c1cc12f020981da1e23bb66c93a6b3c"} Sep 10 17:41:38.944271 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:38.944228 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:41:38.967707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:38.967672 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:38.971133 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:38.971083 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" podStartSLOduration=17.971066824 podStartE2EDuration="17.971066824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:38.970460608 +0000 UTC m=+103.034871340" watchObservedRunningTime="2026-09-10 17:41:38.971066824 +0000 UTC m=+103.035477535" Sep 10 17:41:39.062657 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:39.062569 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:39.163585 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:39.163550 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:39.358804 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:39.358695 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:39.553958 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:39.553928 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:39.949040 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:39.949000 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" event={"ID":"cc2f0d42-0cdf-479e-afbd-fdbfa1672991","Type":"ContainerStarted","Data":"aba0ce61c6495e9c6b71fc0877cc39199a35840f41328b4bd1995531f0d12fd3"} Sep 10 17:41:40.562780 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:40.562745 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:40.957500 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:40.957468 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" event={"ID":"cc2f0d42-0cdf-479e-afbd-fdbfa1672991","Type":"ContainerStarted","Data":"a59ca1ffad31d7270dfc76e3f16d4e68d8483d818d4cd0257ff1e9a7befe36d8"} Sep 10 17:41:40.981520 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:40.981462 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-twqmq" podStartSLOduration=18.043732211 podStartE2EDuration="19.981442386s" totalImagesPullingTime="1.937710175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:40.98060174 +0000 UTC m=+105.045012473" watchObservedRunningTime="2026-09-10 17:41:40.981442386 +0000 UTC m=+105.045853098" Sep 10 17:41:40.988711 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:40.988681 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:41.158694 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:41.158648 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:41.355976 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:41.355848 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:41.757368 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:41.757325 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:42.145026 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:42.144992 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:42.145464 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:42.145047 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:42.145464 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:42.145137 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 17:41:42.145464 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:41:42.145207 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls podName:326a773c-08de-4f4e-af1e-7c46b8b929c9 nodeName:}" failed. No retries permitted until 2026-09-10 17:41:58.14519112 +0000 UTC m=+122.209601808 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w6tmf" (UID: "326a773c-08de-4f4e-af1e-7c46b8b929c9") : secret "insights-runtime-extractor-tls" not found Sep 10 17:41:42.145583 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:42.145507 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/326a773c-08de-4f4e-af1e-7c46b8b929c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:42.551073 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:42.550966 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:42.957734 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:42.957704 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:43.150079 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:43.150046 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:43.353136 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:43.353053 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:43.555500 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:43.555467 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:48.740121 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:48.740086 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:48.839425 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:48.839398 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:49.116581 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.116501 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75946658b-8vvvh"] Sep 10 17:41:49.118502 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.118484 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75946658b-8vvvh" Sep 10 17:41:49.122105 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.122075 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 10 17:41:49.122105 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.122078 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-sfqpn\"" Sep 10 17:41:49.122525 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.122360 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 10 17:41:49.122525 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.122384 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 10 17:41:49.122525 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.122408 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 10 17:41:49.139424 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.139393 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75946658b-8vvvh"] Sep 10 17:41:49.145922 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.145895 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b69d56847-wcx2s"] Sep 10 17:41:49.186162 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.186125 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54f8fb7c56-74zxh"] Sep 10 17:41:49.188039 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.188022 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.216549 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.216517 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54f8fb7c56-74zxh"] Sep 10 17:41:49.284678 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.284644 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:49.302704 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.302674 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-image-registry-private-configuration\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.302822 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.302709 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntm7l\" (UniqueName: \"kubernetes.io/projected/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-kube-api-access-ntm7l\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.302822 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.302741 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-registry-certificates\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.302822 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.302800 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-trusted-ca\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.302954 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.302845 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-bound-sa-token\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.302954 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.302871 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-564bj\" (UniqueName: \"kubernetes.io/projected/053ebca2-7b75-4d97-9fc2-dbe2ba7546b9-kube-api-access-564bj\") pod \"managed-serviceaccount-addon-agent-75946658b-8vvvh\" (UID: \"053ebca2-7b75-4d97-9fc2-dbe2ba7546b9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75946658b-8vvvh" Sep 10 17:41:49.302954 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.302916 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-registry-tls\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.302954 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.302935 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-installation-pull-secrets\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.303091 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.303013 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-ca-trust-extracted\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.303091 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.303034 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/053ebca2-7b75-4d97-9fc2-dbe2ba7546b9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-75946658b-8vvvh\" (UID: \"053ebca2-7b75-4d97-9fc2-dbe2ba7546b9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75946658b-8vvvh" Sep 10 17:41:49.403738 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.403707 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-ca-trust-extracted\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.403914 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.403745 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/053ebca2-7b75-4d97-9fc2-dbe2ba7546b9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-75946658b-8vvvh\" (UID: \"053ebca2-7b75-4d97-9fc2-dbe2ba7546b9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75946658b-8vvvh" Sep 10 17:41:49.403914 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.403880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-image-registry-private-configuration\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.404011 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.403944 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntm7l\" (UniqueName: \"kubernetes.io/projected/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-kube-api-access-ntm7l\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.404011 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.403987 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-registry-certificates\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.404109 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.404011 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-trusted-ca\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.404109 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.404037 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-bound-sa-token\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.404263 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.404227 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-ca-trust-extracted\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.404392 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.404285 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-564bj\" (UniqueName: \"kubernetes.io/projected/053ebca2-7b75-4d97-9fc2-dbe2ba7546b9-kube-api-access-564bj\") pod \"managed-serviceaccount-addon-agent-75946658b-8vvvh\" (UID: \"053ebca2-7b75-4d97-9fc2-dbe2ba7546b9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75946658b-8vvvh" Sep 10 17:41:49.404392 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.404320 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-registry-tls\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.404392 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.404343 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-installation-pull-secrets\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.404906 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.404865 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-registry-certificates\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.405286 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.405266 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-trusted-ca\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.406491 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.406468 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/053ebca2-7b75-4d97-9fc2-dbe2ba7546b9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-75946658b-8vvvh\" (UID: \"053ebca2-7b75-4d97-9fc2-dbe2ba7546b9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75946658b-8vvvh" Sep 10 17:41:49.406697 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.406674 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-installation-pull-secrets\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.406759 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.406726 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-image-registry-private-configuration\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.406814 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.406791 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-registry-tls\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.427937 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.427911 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-bound-sa-token\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.428078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.427969 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-564bj\" (UniqueName: \"kubernetes.io/projected/053ebca2-7b75-4d97-9fc2-dbe2ba7546b9-kube-api-access-564bj\") pod \"managed-serviceaccount-addon-agent-75946658b-8vvvh\" (UID: \"053ebca2-7b75-4d97-9fc2-dbe2ba7546b9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75946658b-8vvvh" Sep 10 17:41:49.429522 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.429491 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntm7l\" (UniqueName: \"kubernetes.io/projected/2f4edcd8-3905-4c92-bfdd-de3e058c21fd-kube-api-access-ntm7l\") pod \"image-registry-54f8fb7c56-74zxh\" (UID: \"2f4edcd8-3905-4c92-bfdd-de3e058c21fd\") " pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.438785 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.438766 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75946658b-8vvvh" Sep 10 17:41:49.442460 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.442427 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:49.496247 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.496186 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:49.712811 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.712730 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75946658b-8vvvh"] Sep 10 17:41:49.715824 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:41:49.715796 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod053ebca2_7b75_4d97_9fc2_dbe2ba7546b9.slice/crio-48ef5133a8b8b5d526f319c4fc78690cebc6aafb10ee502d5984d28a3bcf7e94 WatchSource:0}: Error finding container 48ef5133a8b8b5d526f319c4fc78690cebc6aafb10ee502d5984d28a3bcf7e94: Status 404 returned error can't find the container with id 48ef5133a8b8b5d526f319c4fc78690cebc6aafb10ee502d5984d28a3bcf7e94 Sep 10 17:41:49.774907 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.774855 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54f8fb7c56-74zxh"] Sep 10 17:41:49.777748 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:41:49.777713 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f4edcd8_3905_4c92_bfdd_de3e058c21fd.slice/crio-65a0c44def3910f72fffa407d90f2f3692b78945bcf736a14cecde0e6942c2c5 WatchSource:0}: Error finding container 65a0c44def3910f72fffa407d90f2f3692b78945bcf736a14cecde0e6942c2c5: Status 404 returned error can't find the container with id 65a0c44def3910f72fffa407d90f2f3692b78945bcf736a14cecde0e6942c2c5 Sep 10 17:41:49.985704 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.985611 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75946658b-8vvvh" event={"ID":"053ebca2-7b75-4d97-9fc2-dbe2ba7546b9","Type":"ContainerStarted","Data":"48ef5133a8b8b5d526f319c4fc78690cebc6aafb10ee502d5984d28a3bcf7e94"} Sep 10 17:41:49.987164 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.987137 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" event={"ID":"2f4edcd8-3905-4c92-bfdd-de3e058c21fd","Type":"ContainerStarted","Data":"388226ba093b2323dbd40f44a93dea7efa746c62c41fc04597c48ec60b6ec369"} Sep 10 17:41:49.987280 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.987182 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" event={"ID":"2f4edcd8-3905-4c92-bfdd-de3e058c21fd","Type":"ContainerStarted","Data":"65a0c44def3910f72fffa407d90f2f3692b78945bcf736a14cecde0e6942c2c5"} Sep 10 17:41:49.987316 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:49.987277 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:41:50.010173 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:50.010120 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" podStartSLOduration=1.010101185 podStartE2EDuration="1.010101185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:50.009453877 +0000 UTC m=+114.073864615" watchObservedRunningTime="2026-09-10 17:41:50.010101185 +0000 UTC m=+114.074511886" Sep 10 17:41:52.999710 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:52.999673 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75946658b-8vvvh" event={"ID":"053ebca2-7b75-4d97-9fc2-dbe2ba7546b9","Type":"ContainerStarted","Data":"e6dc402267280de34ac81f3a66318791cbf8d3539f97147088b11030760f49c3"} Sep 10 17:41:53.016536 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:53.016480 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75946658b-8vvvh" podStartSLOduration=1.6695133549999999 podStartE2EDuration="4.01646428s" totalImagesPullingTime="2.346950925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:53.015457222 +0000 UTC m=+117.079867943" watchObservedRunningTime="2026-09-10 17:41:53.01646428 +0000 UTC m=+117.080874990" Sep 10 17:41:53.237198 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:53.237156 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:53.239508 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:53.239475 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e81e5b8d-0807-48b8-8bf7-d31d799e2d40-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-fx8qb\" (UID: \"e81e5b8d-0807-48b8-8bf7-d31d799e2d40\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:53.506014 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:53.505970 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" Sep 10 17:41:53.793489 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:53.793413 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb"] Sep 10 17:41:53.796559 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:41:53.796525 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode81e5b8d_0807_48b8_8bf7_d31d799e2d40.slice/crio-317840b839dfdefb08f56ac2bba20a50758328f3c333b7d69c8b915b0c6cdc15 WatchSource:0}: Error finding container 317840b839dfdefb08f56ac2bba20a50758328f3c333b7d69c8b915b0c6cdc15: Status 404 returned error can't find the container with id 317840b839dfdefb08f56ac2bba20a50758328f3c333b7d69c8b915b0c6cdc15 Sep 10 17:41:54.003977 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:54.003932 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" event={"ID":"e81e5b8d-0807-48b8-8bf7-d31d799e2d40","Type":"ContainerStarted","Data":"317840b839dfdefb08f56ac2bba20a50758328f3c333b7d69c8b915b0c6cdc15"} Sep 10 17:41:55.899786 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:55.899729 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:55.998187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:55.998149 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:56.015249 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:56.015205 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" event={"ID":"e81e5b8d-0807-48b8-8bf7-d31d799e2d40","Type":"ContainerStarted","Data":"05840e3316e570ec8384bdfb3395d2fd464c1a6f06567df2140d661afbdd0dfb"} Sep 10 17:41:56.031100 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:56.031049 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-fx8qb" podStartSLOduration=33.263442205 podStartE2EDuration="35.031033065s" totalImagesPullingTime="1.76759086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:56.030482182 +0000 UTC m=+120.094892903" watchObservedRunningTime="2026-09-10 17:41:56.031033065 +0000 UTC m=+120.095443775" Sep 10 17:41:56.084204 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:56.084172 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:56.760060 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:56.760028 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:56.880820 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:56.880788 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:57.079509 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:57.079431 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:41:58.179404 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:58.179361 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:58.181654 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:58.181630 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/326a773c-08de-4f4e-af1e-7c46b8b929c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w6tmf\" (UID: \"326a773c-08de-4f4e-af1e-7c46b8b929c9\") " pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:58.464634 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:58.464551 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ktmw7\"" Sep 10 17:41:58.473133 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:58.473103 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w6tmf" Sep 10 17:41:58.740280 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:58.740204 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w6tmf"] Sep 10 17:41:58.743322 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:41:58.743289 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod326a773c_08de_4f4e_af1e_7c46b8b929c9.slice/crio-3bb077bdf301d649d036c27ac4e68f72ca49ec16d7ebc679137e77b1bc7c20cb WatchSource:0}: Error finding container 3bb077bdf301d649d036c27ac4e68f72ca49ec16d7ebc679137e77b1bc7c20cb: Status 404 returned error can't find the container with id 3bb077bdf301d649d036c27ac4e68f72ca49ec16d7ebc679137e77b1bc7c20cb Sep 10 17:41:59.027285 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:59.027197 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w6tmf" event={"ID":"326a773c-08de-4f4e-af1e-7c46b8b929c9","Type":"ContainerStarted","Data":"1b44f4a1d5819367d3faa850b22f2a043ba583e8a51f8648441b4506aa652be1"} Sep 10 17:41:59.027427 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:59.027287 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w6tmf" event={"ID":"326a773c-08de-4f4e-af1e-7c46b8b929c9","Type":"ContainerStarted","Data":"3bb077bdf301d649d036c27ac4e68f72ca49ec16d7ebc679137e77b1bc7c20cb"} Sep 10 17:41:59.152597 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:59.152538 2565 patch_prober.go:28] interesting pod/image-registry-7b69d56847-wcx2s container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 10 17:41:59.152762 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:41:59.152622 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" podUID="1683c301-dec1-47bd-b9d8-39d509013d6d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 17:42:00.033599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:00.033561 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w6tmf" event={"ID":"326a773c-08de-4f4e-af1e-7c46b8b929c9","Type":"ContainerStarted","Data":"332a9feeb6edd46523f2669b0ad5756ba76e6f0a120991a60cf8c620e4d102b0"} Sep 10 17:42:00.079190 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:00.079151 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:00.165642 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:00.165609 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:00.871647 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:00.871562 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:01.039051 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:01.039017 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w6tmf" event={"ID":"326a773c-08de-4f4e-af1e-7c46b8b929c9","Type":"ContainerStarted","Data":"fe2f60f0ade32f023b13bd9d61e34543bf4cf3346877ed1a0f2f4d69fef83ab9"} Sep 10 17:42:01.058812 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:01.058755 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-w6tmf" podStartSLOduration=33.29740282 podStartE2EDuration="35.058738997s" totalImagesPullingTime="1.761336177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:41:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:01.05801623 +0000 UTC m=+125.122426963" watchObservedRunningTime="2026-09-10 17:42:01.058738997 +0000 UTC m=+125.123149708" Sep 10 17:42:03.129432 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:03.129397 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:05.233266 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.233230 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:05.240466 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.240435 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs\") pod \"network-metrics-daemon-cvx9n\" (UID: \"b563ccdb-296e-4358-81f7-82dc6bd66435\") " pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:42:05.242693 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.242662 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b563ccdb-296e-4358-81f7-82dc6bd66435-metrics-certs\") pod \"network-metrics-daemon-cvx9n\" (UID: \"b563ccdb-296e-4358-81f7-82dc6bd66435\") " pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:42:05.343298 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.343267 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d22m9\"" Sep 10 17:42:05.351187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.351158 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cvx9n" Sep 10 17:42:05.427772 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.427732 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:05.526936 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.526833 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh"] Sep 10 17:42:05.529912 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.529876 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.532453 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.532429 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 10 17:42:05.532604 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.532469 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 10 17:42:05.532604 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.532594 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 17:42:05.533159 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.533139 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mgzvp\"" Sep 10 17:42:05.533260 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.533139 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 10 17:42:05.533542 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.533526 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:05.546338 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.546313 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh"] Sep 10 17:42:05.558005 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.557976 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jgh7w"] Sep 10 17:42:05.560216 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.560195 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.562511 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.562485 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 17:42:05.562627 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.562517 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 17:42:05.562627 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.562517 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 17:42:05.563093 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.563077 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nplqf\"" Sep 10 17:42:05.617052 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.617016 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cvx9n"] Sep 10 17:42:05.620330 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:42:05.620296 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb563ccdb_296e_4358_81f7_82dc6bd66435.slice/crio-3226da45b594d7386aaf8f5209911936ec6458a98612ddb639c51cae1babbf11 WatchSource:0}: Error finding container 3226da45b594d7386aaf8f5209911936ec6458a98612ddb639c51cae1babbf11: Status 404 returned error can't find the container with id 3226da45b594d7386aaf8f5209911936ec6458a98612ddb639c51cae1babbf11 Sep 10 17:42:05.643323 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643285 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.643323 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643325 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-accelerators-collector-config\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.643543 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643397 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2c7f5e5-0a1d-4e55-a748-36db237c835b-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.643543 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643426 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-tls\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.643543 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643449 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c9354ab8-139a-49ea-970f-07013448edac-root\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.643543 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643481 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sngwn\" (UniqueName: \"kubernetes.io/projected/c2c7f5e5-0a1d-4e55-a748-36db237c835b-kube-api-access-sngwn\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.643543 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643504 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9354ab8-139a-49ea-970f-07013448edac-sys\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.643543 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643522 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-wtmp\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.643799 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643553 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c2c7f5e5-0a1d-4e55-a748-36db237c835b-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.643799 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643574 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2c7f5e5-0a1d-4e55-a748-36db237c835b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.643799 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643591 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-textfile\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.643799 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643610 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5cgr\" (UniqueName: \"kubernetes.io/projected/c9354ab8-139a-49ea-970f-07013448edac-kube-api-access-p5cgr\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.643799 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643657 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9354ab8-139a-49ea-970f-07013448edac-metrics-client-ca\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.643799 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643688 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2c7f5e5-0a1d-4e55-a748-36db237c835b-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.643799 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.643711 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c2c7f5e5-0a1d-4e55-a748-36db237c835b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.647163 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.647140 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:05.744818 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.744780 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c9354ab8-139a-49ea-970f-07013448edac-root\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.744818 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.744816 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sngwn\" (UniqueName: \"kubernetes.io/projected/c2c7f5e5-0a1d-4e55-a748-36db237c835b-kube-api-access-sngwn\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.745054 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.744844 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9354ab8-139a-49ea-970f-07013448edac-sys\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.745054 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.744868 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-wtmp\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.745054 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.744901 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c2c7f5e5-0a1d-4e55-a748-36db237c835b-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.745054 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.744910 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c9354ab8-139a-49ea-970f-07013448edac-root\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.745054 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.744950 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9354ab8-139a-49ea-970f-07013448edac-sys\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.745054 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.744945 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2c7f5e5-0a1d-4e55-a748-36db237c835b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.745054 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.745019 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-textfile\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.745054 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.745033 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-wtmp\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.745054 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.745057 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5cgr\" (UniqueName: \"kubernetes.io/projected/c9354ab8-139a-49ea-970f-07013448edac-kube-api-access-p5cgr\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.745487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.745082 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9354ab8-139a-49ea-970f-07013448edac-metrics-client-ca\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.745487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.745102 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2c7f5e5-0a1d-4e55-a748-36db237c835b-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.745487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.745131 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c2c7f5e5-0a1d-4e55-a748-36db237c835b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.745487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.745219 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.745487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.745244 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-accelerators-collector-config\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.745487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.745302 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2c7f5e5-0a1d-4e55-a748-36db237c835b-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.745487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.745328 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-tls\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.745487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.745335 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c2c7f5e5-0a1d-4e55-a748-36db237c835b-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.745487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.745360 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-textfile\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.745957 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:42:05.745565 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 10 17:42:05.745957 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:42:05.745664 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-tls podName:c9354ab8-139a-49ea-970f-07013448edac nodeName:}" failed. No retries permitted until 2026-09-10 17:42:06.245631181 +0000 UTC m=+130.310041888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-tls") pod "node-exporter-jgh7w" (UID: "c9354ab8-139a-49ea-970f-07013448edac") : secret "node-exporter-tls" not found Sep 10 17:42:05.745957 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.745684 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c9354ab8-139a-49ea-970f-07013448edac-metrics-client-ca\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.746121 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.745958 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-accelerators-collector-config\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.746121 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.746067 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c2c7f5e5-0a1d-4e55-a748-36db237c835b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.746121 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.746090 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2c7f5e5-0a1d-4e55-a748-36db237c835b-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.747631 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.747608 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2c7f5e5-0a1d-4e55-a748-36db237c835b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.747757 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.747733 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c2c7f5e5-0a1d-4e55-a748-36db237c835b-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.747937 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.747901 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:05.747937 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.747930 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.752030 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.752003 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5cgr\" (UniqueName: \"kubernetes.io/projected/c9354ab8-139a-49ea-970f-07013448edac-kube-api-access-p5cgr\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:05.752176 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.752157 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sngwn\" (UniqueName: \"kubernetes.io/projected/c2c7f5e5-0a1d-4e55-a748-36db237c835b-kube-api-access-sngwn\") pod \"kube-state-metrics-74cb57df7-tnvkh\" (UID: \"c2c7f5e5-0a1d-4e55-a748-36db237c835b\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.839389 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.839297 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" Sep 10 17:42:05.849878 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:05.849828 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:06.055191 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:06.055152 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cvx9n" event={"ID":"b563ccdb-296e-4358-81f7-82dc6bd66435","Type":"ContainerStarted","Data":"3226da45b594d7386aaf8f5209911936ec6458a98612ddb639c51cae1babbf11"} Sep 10 17:42:06.109248 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:06.109153 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh"] Sep 10 17:42:06.113747 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:42:06.113716 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2c7f5e5_0a1d_4e55_a748_36db237c835b.slice/crio-9c66b0ffd5e355927b842b882f5815b568e24e6e186119fe3787c6439fe0d5d6 WatchSource:0}: Error finding container 9c66b0ffd5e355927b842b882f5815b568e24e6e186119fe3787c6439fe0d5d6: Status 404 returned error can't find the container with id 9c66b0ffd5e355927b842b882f5815b568e24e6e186119fe3787c6439fe0d5d6 Sep 10 17:42:06.250441 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:06.250410 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-tls\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:06.253288 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:06.253261 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c9354ab8-139a-49ea-970f-07013448edac-node-exporter-tls\") pod \"node-exporter-jgh7w\" (UID: \"c9354ab8-139a-49ea-970f-07013448edac\") " pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:06.294395 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:06.294363 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:06.461120 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:06.461091 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:06.469201 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:06.469174 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jgh7w" Sep 10 17:42:06.526806 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:42:06.526775 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9354ab8_139a_49ea_970f_07013448edac.slice/crio-b6abd4a43483bae45b2d111fa183e3a9cbbe171dcf6ef2a20b4c63f9d4bfe858 WatchSource:0}: Error finding container b6abd4a43483bae45b2d111fa183e3a9cbbe171dcf6ef2a20b4c63f9d4bfe858: Status 404 returned error can't find the container with id b6abd4a43483bae45b2d111fa183e3a9cbbe171dcf6ef2a20b4c63f9d4bfe858 Sep 10 17:42:06.579433 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:06.579396 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:07.061399 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:07.061292 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jgh7w" event={"ID":"c9354ab8-139a-49ea-970f-07013448edac","Type":"ContainerStarted","Data":"b6abd4a43483bae45b2d111fa183e3a9cbbe171dcf6ef2a20b4c63f9d4bfe858"} Sep 10 17:42:07.063058 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:07.062986 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" event={"ID":"c2c7f5e5-0a1d-4e55-a748-36db237c835b","Type":"ContainerStarted","Data":"9c66b0ffd5e355927b842b882f5815b568e24e6e186119fe3787c6439fe0d5d6"} Sep 10 17:42:07.065404 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:07.065375 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cvx9n" event={"ID":"b563ccdb-296e-4358-81f7-82dc6bd66435","Type":"ContainerStarted","Data":"92a3759665656167e76bc2d52a81840b247405a1cb68612130133d47a8644700"} Sep 10 17:42:07.065519 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:07.065432 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cvx9n" event={"ID":"b563ccdb-296e-4358-81f7-82dc6bd66435","Type":"ContainerStarted","Data":"275ad4cea6ba7e0464f10d47e03a329bdc0efdc503acc031f29955e9bed03325"} Sep 10 17:42:07.081901 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:07.081795 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cvx9n" podStartSLOduration=129.157727007 podStartE2EDuration="2m10.081772282s" totalImagesPullingTime="924.045275ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:07.080691121 +0000 UTC m=+131.145101852" watchObservedRunningTime="2026-09-10 17:42:07.081772282 +0000 UTC m=+131.146182994" Sep 10 17:42:07.185147 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:07.185110 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:07.287169 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:07.287130 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:07.389881 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:07.389852 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:08.071109 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:08.071014 2565 generic.go:356] "Generic (PLEG): container finished" podID="c9354ab8-139a-49ea-970f-07013448edac" containerID="7e2ca57674c590ea0106cb2b25883ee619575046b427bf12218272383eef7b8f" exitCode=0 Sep 10 17:42:08.071271 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:08.071104 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jgh7w" event={"ID":"c9354ab8-139a-49ea-970f-07013448edac","Type":"ContainerDied","Data":"7e2ca57674c590ea0106cb2b25883ee619575046b427bf12218272383eef7b8f"} Sep 10 17:42:08.073596 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:08.073568 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" event={"ID":"c2c7f5e5-0a1d-4e55-a748-36db237c835b","Type":"ContainerStarted","Data":"c4a38781ea3c5b0861acc819f43bb9e730de94c0d434845733bac4f70f76d8dd"} Sep 10 17:42:08.073693 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:08.073626 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" event={"ID":"c2c7f5e5-0a1d-4e55-a748-36db237c835b","Type":"ContainerStarted","Data":"9b21bf3e308e02553ffa4db851281ba9fa81cf093d4c34eb9861d9ff6766f2e2"} Sep 10 17:42:08.073693 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:08.073656 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" event={"ID":"c2c7f5e5-0a1d-4e55-a748-36db237c835b","Type":"ContainerStarted","Data":"d233ed8584999c3211dc47bcb7cae119897926db049ee0ecce5b753eb2c88dbc"} Sep 10 17:42:08.124955 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:08.124874 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-tnvkh" podStartSLOduration=1.591851534 podStartE2EDuration="3.124857349s" totalImagesPullingTime="1.533005815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:08.123799208 +0000 UTC m=+132.188209930" watchObservedRunningTime="2026-09-10 17:42:08.124857349 +0000 UTC m=+132.189268060" Sep 10 17:42:08.484587 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:08.484549 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:08.605491 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:08.605456 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:09.079698 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:09.079657 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jgh7w" event={"ID":"c9354ab8-139a-49ea-970f-07013448edac","Type":"ContainerStarted","Data":"632d395b7abdbce0885220d15a6f7feb973f592c385b897bd0ca77e21291fb75"} Sep 10 17:42:09.079871 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:09.079719 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jgh7w" event={"ID":"c9354ab8-139a-49ea-970f-07013448edac","Type":"ContainerStarted","Data":"1311f3e2f2523c3236f276e37c52fc177a816f859db16d9815effbb8dce6d33a"} Sep 10 17:42:09.107299 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:09.107247 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jgh7w" podStartSLOduration=2.906927618 podStartE2EDuration="4.107230185s" totalImagesPullingTime="1.118491391s" totalInitContainerRuntime="81.811176ms" isStatefulPod=true podCreationTimestamp="2026-09-10 17:42:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:09.104920503 +0000 UTC m=+133.169331231" watchObservedRunningTime="2026-09-10 17:42:09.107230185 +0000 UTC m=+133.171640898" Sep 10 17:42:09.109418 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:09.109393 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:09.150306 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:09.150275 2565 patch_prober.go:28] interesting pod/image-registry-7b69d56847-wcx2s container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 10 17:42:09.150442 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:09.150321 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" podUID="1683c301-dec1-47bd-b9d8-39d509013d6d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 17:42:09.385022 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:09.384980 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:09.502179 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:09.502139 2565 patch_prober.go:28] interesting pod/image-registry-54f8fb7c56-74zxh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 10 17:42:09.502602 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:09.502198 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" podUID="2f4edcd8-3905-4c92-bfdd-de3e058c21fd" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 17:42:10.276907 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.275963 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5"] Sep 10 17:42:10.279396 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.279373 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5" Sep 10 17:42:10.281275 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.281254 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-x9zsn\"" Sep 10 17:42:10.282087 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.282066 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 10 17:42:10.289834 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.289811 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5"] Sep 10 17:42:10.299789 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.299765 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:10.387164 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.387126 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tphj\" (UniqueName: \"kubernetes.io/projected/b8f58d4b-46b9-4fe9-98dc-3ff4177eb695-kube-api-access-5tphj\") pod \"monitoring-plugin-85bbbbbf4-gkwr5\" (UID: \"b8f58d4b-46b9-4fe9-98dc-3ff4177eb695\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5" Sep 10 17:42:10.387306 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.387172 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8f58d4b-46b9-4fe9-98dc-3ff4177eb695-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-gkwr5\" (UID: \"b8f58d4b-46b9-4fe9-98dc-3ff4177eb695\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5" Sep 10 17:42:10.488241 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.488199 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8f58d4b-46b9-4fe9-98dc-3ff4177eb695-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-gkwr5\" (UID: \"b8f58d4b-46b9-4fe9-98dc-3ff4177eb695\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5" Sep 10 17:42:10.488409 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.488289 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tphj\" (UniqueName: \"kubernetes.io/projected/b8f58d4b-46b9-4fe9-98dc-3ff4177eb695-kube-api-access-5tphj\") pod \"monitoring-plugin-85bbbbbf4-gkwr5\" (UID: \"b8f58d4b-46b9-4fe9-98dc-3ff4177eb695\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5" Sep 10 17:42:10.488409 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:42:10.488351 2565 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 10 17:42:10.488483 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:42:10.488414 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8f58d4b-46b9-4fe9-98dc-3ff4177eb695-monitoring-plugin-cert podName:b8f58d4b-46b9-4fe9-98dc-3ff4177eb695 nodeName:}" failed. No retries permitted until 2026-09-10 17:42:10.988398345 +0000 UTC m=+135.052809034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b8f58d4b-46b9-4fe9-98dc-3ff4177eb695-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-gkwr5" (UID: "b8f58d4b-46b9-4fe9-98dc-3ff4177eb695") : secret "monitoring-plugin-cert" not found Sep 10 17:42:10.499667 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.499633 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tphj\" (UniqueName: \"kubernetes.io/projected/b8f58d4b-46b9-4fe9-98dc-3ff4177eb695-kube-api-access-5tphj\") pod \"monitoring-plugin-85bbbbbf4-gkwr5\" (UID: \"b8f58d4b-46b9-4fe9-98dc-3ff4177eb695\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5" Sep 10 17:42:10.580709 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.580616 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:10.992050 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.992004 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8f58d4b-46b9-4fe9-98dc-3ff4177eb695-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-gkwr5\" (UID: \"b8f58d4b-46b9-4fe9-98dc-3ff4177eb695\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5" Sep 10 17:42:10.994879 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.994841 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8f58d4b-46b9-4fe9-98dc-3ff4177eb695-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-gkwr5\" (UID: \"b8f58d4b-46b9-4fe9-98dc-3ff4177eb695\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5" Sep 10 17:42:10.996332 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.996304 2565 patch_prober.go:28] interesting pod/image-registry-54f8fb7c56-74zxh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 10 17:42:10.996461 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:10.996358 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" podUID="2f4edcd8-3905-4c92-bfdd-de3e058c21fd" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 17:42:11.188708 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:11.188673 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5" Sep 10 17:42:11.457347 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:11.457317 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5"] Sep 10 17:42:11.460386 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:42:11.460361 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8f58d4b_46b9_4fe9_98dc_3ff4177eb695.slice/crio-a7d3cd1c51dfb08b4942d8c384ec4b00ecddc676c4019e4f62e0ee927d6a5f3c WatchSource:0}: Error finding container a7d3cd1c51dfb08b4942d8c384ec4b00ecddc676c4019e4f62e0ee927d6a5f3c: Status 404 returned error can't find the container with id a7d3cd1c51dfb08b4942d8c384ec4b00ecddc676c4019e4f62e0ee927d6a5f3c Sep 10 17:42:11.758549 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:11.758472 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:12.093823 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:12.093729 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5" event={"ID":"b8f58d4b-46b9-4fe9-98dc-3ff4177eb695","Type":"ContainerStarted","Data":"a7d3cd1c51dfb08b4942d8c384ec4b00ecddc676c4019e4f62e0ee927d6a5f3c"} Sep 10 17:42:13.098488 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:13.098448 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5" event={"ID":"b8f58d4b-46b9-4fe9-98dc-3ff4177eb695","Type":"ContainerStarted","Data":"69ba22303eecd5aaa481dec2576cb15bb10b068fdcf2bdc824a0448bf1c03a1d"} Sep 10 17:42:13.098987 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:13.098670 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5" Sep 10 17:42:13.103550 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:13.103526 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5" Sep 10 17:42:13.116654 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:13.116591 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-gkwr5" podStartSLOduration=1.6707931710000001 podStartE2EDuration="3.116570816s" totalImagesPullingTime="1.445777645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:13.116184615 +0000 UTC m=+137.180595348" watchObservedRunningTime="2026-09-10 17:42:13.116570816 +0000 UTC m=+137.180981529" Sep 10 17:42:14.165635 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.165574 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" podUID="1683c301-dec1-47bd-b9d8-39d509013d6d" containerName="registry" containerID="cri-o://10b1e5a223d14d0d1538a57f2c7c7e1c2c1cc12f020981da1e23bb66c93a6b3c" gracePeriod=30 Sep 10 17:42:14.396718 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.396691 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:42:14.523626 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.523538 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1683c301-dec1-47bd-b9d8-39d509013d6d-installation-pull-secrets\") pod \"1683c301-dec1-47bd-b9d8-39d509013d6d\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " Sep 10 17:42:14.523626 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.523577 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1683c301-dec1-47bd-b9d8-39d509013d6d-trusted-ca\") pod \"1683c301-dec1-47bd-b9d8-39d509013d6d\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " Sep 10 17:42:14.523626 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.523600 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls\") pod \"1683c301-dec1-47bd-b9d8-39d509013d6d\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " Sep 10 17:42:14.523626 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.523626 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-bound-sa-token\") pod \"1683c301-dec1-47bd-b9d8-39d509013d6d\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " Sep 10 17:42:14.523984 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.523688 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1683c301-dec1-47bd-b9d8-39d509013d6d-image-registry-private-configuration\") pod \"1683c301-dec1-47bd-b9d8-39d509013d6d\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " Sep 10 17:42:14.523984 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.523745 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1683c301-dec1-47bd-b9d8-39d509013d6d-ca-trust-extracted\") pod \"1683c301-dec1-47bd-b9d8-39d509013d6d\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " Sep 10 17:42:14.523984 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.523775 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-certificates\") pod \"1683c301-dec1-47bd-b9d8-39d509013d6d\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " Sep 10 17:42:14.523984 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.523803 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pr6m9\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-kube-api-access-pr6m9\") pod \"1683c301-dec1-47bd-b9d8-39d509013d6d\" (UID: \"1683c301-dec1-47bd-b9d8-39d509013d6d\") " Sep 10 17:42:14.524188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.524126 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1683c301-dec1-47bd-b9d8-39d509013d6d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1683c301-dec1-47bd-b9d8-39d509013d6d" (UID: "1683c301-dec1-47bd-b9d8-39d509013d6d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:42:14.524476 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.524423 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1683c301-dec1-47bd-b9d8-39d509013d6d" (UID: "1683c301-dec1-47bd-b9d8-39d509013d6d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:42:14.526526 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.526492 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1683c301-dec1-47bd-b9d8-39d509013d6d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1683c301-dec1-47bd-b9d8-39d509013d6d" (UID: "1683c301-dec1-47bd-b9d8-39d509013d6d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:42:14.526526 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.526503 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1683c301-dec1-47bd-b9d8-39d509013d6d" (UID: "1683c301-dec1-47bd-b9d8-39d509013d6d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:42:14.526678 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.526564 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1683c301-dec1-47bd-b9d8-39d509013d6d" (UID: "1683c301-dec1-47bd-b9d8-39d509013d6d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:42:14.526678 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.526582 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-kube-api-access-pr6m9" (OuterVolumeSpecName: "kube-api-access-pr6m9") pod "1683c301-dec1-47bd-b9d8-39d509013d6d" (UID: "1683c301-dec1-47bd-b9d8-39d509013d6d"). InnerVolumeSpecName "kube-api-access-pr6m9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:42:14.526678 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.526650 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1683c301-dec1-47bd-b9d8-39d509013d6d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1683c301-dec1-47bd-b9d8-39d509013d6d" (UID: "1683c301-dec1-47bd-b9d8-39d509013d6d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:42:14.532685 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.532656 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1683c301-dec1-47bd-b9d8-39d509013d6d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1683c301-dec1-47bd-b9d8-39d509013d6d" (UID: "1683c301-dec1-47bd-b9d8-39d509013d6d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:42:14.624808 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.624769 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1683c301-dec1-47bd-b9d8-39d509013d6d-image-registry-private-configuration\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:42:14.624808 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.624799 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1683c301-dec1-47bd-b9d8-39d509013d6d-ca-trust-extracted\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:42:14.624808 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.624809 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-certificates\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:42:14.624808 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.624817 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr6m9\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-kube-api-access-pr6m9\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:42:14.625086 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.624827 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1683c301-dec1-47bd-b9d8-39d509013d6d-installation-pull-secrets\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:42:14.625086 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.624836 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1683c301-dec1-47bd-b9d8-39d509013d6d-trusted-ca\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:42:14.625086 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.624844 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-registry-tls\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:42:14.625086 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:14.624851 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1683c301-dec1-47bd-b9d8-39d509013d6d-bound-sa-token\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:42:15.106132 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:15.106094 2565 generic.go:356] "Generic (PLEG): container finished" podID="1683c301-dec1-47bd-b9d8-39d509013d6d" containerID="10b1e5a223d14d0d1538a57f2c7c7e1c2c1cc12f020981da1e23bb66c93a6b3c" exitCode=0 Sep 10 17:42:15.106321 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:15.106137 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" Sep 10 17:42:15.106321 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:15.106179 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" event={"ID":"1683c301-dec1-47bd-b9d8-39d509013d6d","Type":"ContainerDied","Data":"10b1e5a223d14d0d1538a57f2c7c7e1c2c1cc12f020981da1e23bb66c93a6b3c"} Sep 10 17:42:15.106321 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:15.106231 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b69d56847-wcx2s" event={"ID":"1683c301-dec1-47bd-b9d8-39d509013d6d","Type":"ContainerDied","Data":"b9a8d2d553b260b376e77051fa4305f3ed91c73e35642c4cdbc181e6dabd0a02"} Sep 10 17:42:15.106321 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:15.106251 2565 scope.go:117] "RemoveContainer" containerID="10b1e5a223d14d0d1538a57f2c7c7e1c2c1cc12f020981da1e23bb66c93a6b3c" Sep 10 17:42:15.114916 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:15.114896 2565 scope.go:117] "RemoveContainer" containerID="10b1e5a223d14d0d1538a57f2c7c7e1c2c1cc12f020981da1e23bb66c93a6b3c" Sep 10 17:42:15.115216 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:42:15.115196 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10b1e5a223d14d0d1538a57f2c7c7e1c2c1cc12f020981da1e23bb66c93a6b3c\": container with ID starting with 10b1e5a223d14d0d1538a57f2c7c7e1c2c1cc12f020981da1e23bb66c93a6b3c not found: ID does not exist" containerID="10b1e5a223d14d0d1538a57f2c7c7e1c2c1cc12f020981da1e23bb66c93a6b3c" Sep 10 17:42:15.115269 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:15.115226 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10b1e5a223d14d0d1538a57f2c7c7e1c2c1cc12f020981da1e23bb66c93a6b3c"} err="failed to get container status \"10b1e5a223d14d0d1538a57f2c7c7e1c2c1cc12f020981da1e23bb66c93a6b3c\": rpc error: code = NotFound desc = could not find container \"10b1e5a223d14d0d1538a57f2c7c7e1c2c1cc12f020981da1e23bb66c93a6b3c\": container with ID starting with 10b1e5a223d14d0d1538a57f2c7c7e1c2c1cc12f020981da1e23bb66c93a6b3c not found: ID does not exist" Sep 10 17:42:15.126232 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:15.126212 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b69d56847-wcx2s"] Sep 10 17:42:15.132141 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:15.132121 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7b69d56847-wcx2s"] Sep 10 17:42:16.435633 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:16.435603 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1683c301-dec1-47bd-b9d8-39d509013d6d" path="/var/lib/kubelet/pods/1683c301-dec1-47bd-b9d8-39d509013d6d/volumes" Sep 10 17:42:19.500976 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:19.500937 2565 patch_prober.go:28] interesting pod/image-registry-54f8fb7c56-74zxh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 10 17:42:19.501329 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:19.500993 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" podUID="2f4edcd8-3905-4c92-bfdd-de3e058c21fd" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 17:42:20.820006 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:20.819971 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-pgwlp"] Sep 10 17:42:20.820380 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:20.820231 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1683c301-dec1-47bd-b9d8-39d509013d6d" containerName="registry" Sep 10 17:42:20.820380 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:20.820241 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1683c301-dec1-47bd-b9d8-39d509013d6d" containerName="registry" Sep 10 17:42:20.820380 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:20.820286 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1683c301-dec1-47bd-b9d8-39d509013d6d" containerName="registry" Sep 10 17:42:20.822206 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:20.822190 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-pgwlp" Sep 10 17:42:20.826555 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:20.826532 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 17:42:20.826685 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:20.826608 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 17:42:20.827047 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:20.827033 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-2knc7\"" Sep 10 17:42:20.840459 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:20.840431 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-pgwlp"] Sep 10 17:42:20.855658 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:20.855626 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:20.940554 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:20.940519 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:20.975433 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:20.975403 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmkvd\" (UniqueName: \"kubernetes.io/projected/d4f61f63-0da1-4805-9490-f82e8aec75ea-kube-api-access-fmkvd\") pod \"downloads-85d98c998f-pgwlp\" (UID: \"d4f61f63-0da1-4805-9490-f82e8aec75ea\") " pod="openshift-console/downloads-85d98c998f-pgwlp" Sep 10 17:42:20.996204 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:20.996170 2565 patch_prober.go:28] interesting pod/image-registry-54f8fb7c56-74zxh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 10 17:42:20.996343 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:20.996220 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" podUID="2f4edcd8-3905-4c92-bfdd-de3e058c21fd" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 17:42:21.076612 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:21.076510 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmkvd\" (UniqueName: \"kubernetes.io/projected/d4f61f63-0da1-4805-9490-f82e8aec75ea-kube-api-access-fmkvd\") pod \"downloads-85d98c998f-pgwlp\" (UID: \"d4f61f63-0da1-4805-9490-f82e8aec75ea\") " pod="openshift-console/downloads-85d98c998f-pgwlp" Sep 10 17:42:21.086382 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:21.086356 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmkvd\" (UniqueName: \"kubernetes.io/projected/d4f61f63-0da1-4805-9490-f82e8aec75ea-kube-api-access-fmkvd\") pod \"downloads-85d98c998f-pgwlp\" (UID: \"d4f61f63-0da1-4805-9490-f82e8aec75ea\") " pod="openshift-console/downloads-85d98c998f-pgwlp" Sep 10 17:42:21.131045 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:21.131011 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-pgwlp" Sep 10 17:42:21.137767 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:21.137736 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:21.258298 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:21.258260 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:21.338018 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:21.337945 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:21.400701 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:21.400665 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-pgwlp"] Sep 10 17:42:21.403421 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:42:21.403381 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f61f63_0da1_4805_9490_f82e8aec75ea.slice/crio-4c14cda57fd1b8eae679a550fae261fc2605ba895587e63cceab77125cc1fe6a WatchSource:0}: Error finding container 4c14cda57fd1b8eae679a550fae261fc2605ba895587e63cceab77125cc1fe6a: Status 404 returned error can't find the container with id 4c14cda57fd1b8eae679a550fae261fc2605ba895587e63cceab77125cc1fe6a Sep 10 17:42:21.649193 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:21.649159 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:21.750509 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:21.750478 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:21.902235 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:21.902158 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:22.058178 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:22.058144 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:22.133452 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:22.133419 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-pgwlp" event={"ID":"d4f61f63-0da1-4805-9490-f82e8aec75ea","Type":"ContainerStarted","Data":"4c14cda57fd1b8eae679a550fae261fc2605ba895587e63cceab77125cc1fe6a"} Sep 10 17:42:22.160015 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:22.159910 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:22.469793 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:22.469718 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:22.660322 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:22.660291 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:22.761942 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:22.761862 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:23.879013 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:23.878976 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:26.129261 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:26.129219 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:26.193254 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:26.193224 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:29.501877 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:29.501836 2565 patch_prober.go:28] interesting pod/image-registry-54f8fb7c56-74zxh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 10 17:42:29.502351 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:29.501939 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" podUID="2f4edcd8-3905-4c92-bfdd-de3e058c21fd" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 17:42:29.502351 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:29.501992 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:42:29.503002 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:29.502545 2565 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"388226ba093b2323dbd40f44a93dea7efa746c62c41fc04597c48ec60b6ec369"} pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" containerMessage="Container registry failed liveness probe, will be restarted" Sep 10 17:42:29.507242 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:29.507216 2565 patch_prober.go:28] interesting pod/image-registry-54f8fb7c56-74zxh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 10 17:42:29.507368 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:29.507283 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" podUID="2f4edcd8-3905-4c92-bfdd-de3e058c21fd" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 17:42:30.878564 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:30.877274 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84cf7655d-dst5b"] Sep 10 17:42:30.880053 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:30.880025 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:30.882796 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:30.882766 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 17:42:30.882940 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:30.882850 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 17:42:30.882940 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:30.882859 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 17:42:30.883041 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:30.882765 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 17:42:30.883041 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:30.882765 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9s9rn\"" Sep 10 17:42:30.883041 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:30.882765 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 17:42:30.889315 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:30.889093 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84cf7655d-dst5b"] Sep 10 17:42:31.069530 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.069496 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q75zg\" (UniqueName: \"kubernetes.io/projected/d738c6a6-4752-45be-b309-227ed99aa11b-kube-api-access-q75zg\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.069726 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.069564 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d738c6a6-4752-45be-b309-227ed99aa11b-console-serving-cert\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.069726 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.069625 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d738c6a6-4752-45be-b309-227ed99aa11b-console-oauth-config\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.069726 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.069649 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-console-config\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.069726 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.069702 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-service-ca\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.069946 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.069734 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-oauth-serving-cert\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.170948 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.170908 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d738c6a6-4752-45be-b309-227ed99aa11b-console-serving-cert\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.171134 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.170980 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d738c6a6-4752-45be-b309-227ed99aa11b-console-oauth-config\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.171134 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.171007 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-console-config\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.171134 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.171053 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-service-ca\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.171134 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.171080 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-oauth-serving-cert\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.171134 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.171119 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q75zg\" (UniqueName: \"kubernetes.io/projected/d738c6a6-4752-45be-b309-227ed99aa11b-kube-api-access-q75zg\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.171806 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.171783 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-console-config\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.171937 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.171908 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-service-ca\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.171989 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.171957 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-oauth-serving-cert\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.173725 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.173705 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d738c6a6-4752-45be-b309-227ed99aa11b-console-oauth-config\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.174027 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.174003 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d738c6a6-4752-45be-b309-227ed99aa11b-console-serving-cert\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.179963 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.179939 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q75zg\" (UniqueName: \"kubernetes.io/projected/d738c6a6-4752-45be-b309-227ed99aa11b-kube-api-access-q75zg\") pod \"console-84cf7655d-dst5b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:31.192767 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:31.192736 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:33.113661 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:33.113619 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:33.231549 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:33.231511 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:34.331961 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:42:34.331918 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gnq56" podUID="25d7cbef-d614-40ec-bf7c-517c5e752cea" Sep 10 17:42:35.185597 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:35.185569 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gnq56" Sep 10 17:42:36.281634 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:36.281565 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:37.608790 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:37.608661 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84cf7655d-dst5b"] Sep 10 17:42:37.611578 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:42:37.611541 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd738c6a6_4752_45be_b309_227ed99aa11b.slice/crio-7e4c21e9934900ddc97a0ee3166b08e8e52467ca0d50c3276e19b75680738000 WatchSource:0}: Error finding container 7e4c21e9934900ddc97a0ee3166b08e8e52467ca0d50c3276e19b75680738000: Status 404 returned error can't find the container with id 7e4c21e9934900ddc97a0ee3166b08e8e52467ca0d50c3276e19b75680738000 Sep 10 17:42:38.201617 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:38.201512 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-pgwlp" event={"ID":"d4f61f63-0da1-4805-9490-f82e8aec75ea","Type":"ContainerStarted","Data":"b696539490f00a849eb91fefff92f3e47014c8fa1f0b0d14545a495fef31ab67"} Sep 10 17:42:38.201617 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:38.201587 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-pgwlp" Sep 10 17:42:38.204596 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:38.204562 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84cf7655d-dst5b" event={"ID":"d738c6a6-4752-45be-b309-227ed99aa11b","Type":"ContainerStarted","Data":"7e4c21e9934900ddc97a0ee3166b08e8e52467ca0d50c3276e19b75680738000"} Sep 10 17:42:38.212401 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:38.212342 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-pgwlp" Sep 10 17:42:38.226657 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:38.226596 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-pgwlp" podStartSLOduration=2.219447337 podStartE2EDuration="18.226574648s" totalImagesPullingTime="16.007127311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:38.226199796 +0000 UTC m=+162.290610529" watchObservedRunningTime="2026-09-10 17:42:38.226574648 +0000 UTC m=+162.290985361" Sep 10 17:42:38.874692 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:38.874653 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:38.975472 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:38.975432 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:39.039217 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:39.039110 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:39.242800 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:39.242765 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:42:39.246413 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:39.246347 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/25d7cbef-d614-40ec-bf7c-517c5e752cea-metrics-tls\") pod \"dns-default-gnq56\" (UID: \"25d7cbef-d614-40ec-bf7c-517c5e752cea\") " pod="openshift-dns/dns-default-gnq56" Sep 10 17:42:39.388993 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:39.388962 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dwwmp\"" Sep 10 17:42:39.396781 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:39.396745 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gnq56" Sep 10 17:42:39.509718 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:39.509074 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:42:39.634981 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:39.634937 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:39.693804 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:39.693768 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:39.714379 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:39.714350 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gnq56"] Sep 10 17:42:39.815719 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:39.815626 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:40.085837 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:40.085748 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:40.186107 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:40.186075 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:40.286603 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:40.286570 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:40.827578 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:42:40.827537 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25d7cbef_d614_40ec_bf7c_517c5e752cea.slice/crio-773f363e359c021cf9bd9658d5878e730c357910de71b5d44dc318782ecd39ef WatchSource:0}: Error finding container 773f363e359c021cf9bd9658d5878e730c357910de71b5d44dc318782ecd39ef: Status 404 returned error can't find the container with id 773f363e359c021cf9bd9658d5878e730c357910de71b5d44dc318782ecd39ef Sep 10 17:42:41.221679 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:41.221636 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84cf7655d-dst5b" event={"ID":"d738c6a6-4752-45be-b309-227ed99aa11b","Type":"ContainerStarted","Data":"8643171e9c30530e22a9cbbebe6eaea27fe3573d89f393525bb4fa78af5fd138"} Sep 10 17:42:41.223267 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:41.223235 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gnq56" event={"ID":"25d7cbef-d614-40ec-bf7c-517c5e752cea","Type":"ContainerStarted","Data":"773f363e359c021cf9bd9658d5878e730c357910de71b5d44dc318782ecd39ef"} Sep 10 17:42:41.241872 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:41.241805 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84cf7655d-dst5b" podStartSLOduration=7.7682332899999995 podStartE2EDuration="11.241782107s" totalImagesPullingTime="3.473548817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:41.24152054 +0000 UTC m=+165.305931271" watchObservedRunningTime="2026-09-10 17:42:41.241782107 +0000 UTC m=+165.306192813" Sep 10 17:42:43.359640 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:43.359607 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:43.440203 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:43.440168 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:43.858392 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:43.858309 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84cf7655d-dst5b"] Sep 10 17:42:44.238367 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:44.238328 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gnq56" event={"ID":"25d7cbef-d614-40ec-bf7c-517c5e752cea","Type":"ContainerStarted","Data":"73895ad23099836b52c800412fbfac989b73bd2584f58d1ca9ba51d084d515b3"} Sep 10 17:42:44.238564 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:44.238389 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gnq56" event={"ID":"25d7cbef-d614-40ec-bf7c-517c5e752cea","Type":"ContainerStarted","Data":"4d36d07a3a5b926461750b1819a12125a94e2ed7689094f559fd9c1b1944bb05"} Sep 10 17:42:44.238564 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:44.238483 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gnq56" Sep 10 17:42:44.256595 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:44.256532 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gnq56" podStartSLOduration=130.832324156 podStartE2EDuration="2m13.256510458s" totalImagesPullingTime="2.424186302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:44.254878966 +0000 UTC m=+168.319289710" watchObservedRunningTime="2026-09-10 17:42:44.256510458 +0000 UTC m=+168.320921179" Sep 10 17:42:45.096193 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:45.096155 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:45.164323 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:45.164288 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:46.705074 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:46.705037 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:46.810330 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:46.810290 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:48.683771 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:48.683729 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:48.785728 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:48.785695 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:50.262824 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:50.262740 2565 generic.go:356] "Generic (PLEG): container finished" podID="06559b57-0c49-4988-87ab-01fde08339dd" containerID="939512c3b3435fe06cd698b92491a065237f3bd0d57b2ce19c63ce5a71e922f0" exitCode=0 Sep 10 17:42:50.262824 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:50.262801 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" event={"ID":"06559b57-0c49-4988-87ab-01fde08339dd","Type":"ContainerDied","Data":"939512c3b3435fe06cd698b92491a065237f3bd0d57b2ce19c63ce5a71e922f0"} Sep 10 17:42:50.263268 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:50.263127 2565 scope.go:117] "RemoveContainer" containerID="939512c3b3435fe06cd698b92491a065237f3bd0d57b2ce19c63ce5a71e922f0" Sep 10 17:42:51.193721 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:51.193669 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:42:51.268622 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:51.268583 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-7tr6s" event={"ID":"06559b57-0c49-4988-87ab-01fde08339dd","Type":"ContainerStarted","Data":"314a72a3459e819de12a75852f8b366e0688501d74fe783b0cc645ba726a8cea"} Sep 10 17:42:52.455340 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:52.455308 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:52.556674 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:52.556641 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:54.245789 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:54.245756 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gnq56" Sep 10 17:42:54.524898 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:54.524797 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" podUID="2f4edcd8-3905-4c92-bfdd-de3e058c21fd" containerName="registry" containerID="cri-o://388226ba093b2323dbd40f44a93dea7efa746c62c41fc04597c48ec60b6ec369" gracePeriod=30 Sep 10 17:42:56.288022 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:56.287990 2565 generic.go:356] "Generic (PLEG): container finished" podID="2f4edcd8-3905-4c92-bfdd-de3e058c21fd" containerID="388226ba093b2323dbd40f44a93dea7efa746c62c41fc04597c48ec60b6ec369" exitCode=0 Sep 10 17:42:56.288481 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:56.288047 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" event={"ID":"2f4edcd8-3905-4c92-bfdd-de3e058c21fd","Type":"ContainerDied","Data":"388226ba093b2323dbd40f44a93dea7efa746c62c41fc04597c48ec60b6ec369"} Sep 10 17:42:56.288481 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:56.288082 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" event={"ID":"2f4edcd8-3905-4c92-bfdd-de3e058c21fd","Type":"ContainerStarted","Data":"69a2cdb5371fd14c8d43b57ac40db022a5b44918e9e18b6bddb7bfb162d250cb"} Sep 10 17:42:56.288481 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:56.288157 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:42:56.598814 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:56.598727 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sx5vt_5e2f1a90-af6b-49cc-9466-57b7848dcaf3/serve-healthcheck-canary/0.log" Sep 10 17:42:56.645801 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:56.645767 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:56.722769 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:56.722738 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:56.923612 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:56.923581 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:57.034639 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:57.034607 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:42:57.100336 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:42:57.100303 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:03.409904 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:03.409853 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:03.493001 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:03.492966 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:03.693238 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:03.693162 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:03.791735 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:03.791697 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:08.882581 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:08.882498 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84cf7655d-dst5b" podUID="d738c6a6-4752-45be-b309-227ed99aa11b" containerName="console" containerID="cri-o://8643171e9c30530e22a9cbbebe6eaea27fe3573d89f393525bb4fa78af5fd138" gracePeriod=15 Sep 10 17:43:09.184556 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.184124 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:43:09.300072 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.300040 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-service-ca\") pod \"d738c6a6-4752-45be-b309-227ed99aa11b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " Sep 10 17:43:09.300072 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.300080 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q75zg\" (UniqueName: \"kubernetes.io/projected/d738c6a6-4752-45be-b309-227ed99aa11b-kube-api-access-q75zg\") pod \"d738c6a6-4752-45be-b309-227ed99aa11b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " Sep 10 17:43:09.300347 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.300107 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d738c6a6-4752-45be-b309-227ed99aa11b-console-oauth-config\") pod \"d738c6a6-4752-45be-b309-227ed99aa11b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " Sep 10 17:43:09.300347 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.300150 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-console-config\") pod \"d738c6a6-4752-45be-b309-227ed99aa11b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " Sep 10 17:43:09.300347 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.300174 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d738c6a6-4752-45be-b309-227ed99aa11b-console-serving-cert\") pod \"d738c6a6-4752-45be-b309-227ed99aa11b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " Sep 10 17:43:09.300347 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.300264 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-oauth-serving-cert\") pod \"d738c6a6-4752-45be-b309-227ed99aa11b\" (UID: \"d738c6a6-4752-45be-b309-227ed99aa11b\") " Sep 10 17:43:09.300550 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.300444 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-service-ca" (OuterVolumeSpecName: "service-ca") pod "d738c6a6-4752-45be-b309-227ed99aa11b" (UID: "d738c6a6-4752-45be-b309-227ed99aa11b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:43:09.300606 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.300538 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-console-config" (OuterVolumeSpecName: "console-config") pod "d738c6a6-4752-45be-b309-227ed99aa11b" (UID: "d738c6a6-4752-45be-b309-227ed99aa11b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:43:09.300709 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.300680 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d738c6a6-4752-45be-b309-227ed99aa11b" (UID: "d738c6a6-4752-45be-b309-227ed99aa11b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:43:09.302461 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.302428 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d738c6a6-4752-45be-b309-227ed99aa11b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d738c6a6-4752-45be-b309-227ed99aa11b" (UID: "d738c6a6-4752-45be-b309-227ed99aa11b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:43:09.302572 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.302512 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d738c6a6-4752-45be-b309-227ed99aa11b-kube-api-access-q75zg" (OuterVolumeSpecName: "kube-api-access-q75zg") pod "d738c6a6-4752-45be-b309-227ed99aa11b" (UID: "d738c6a6-4752-45be-b309-227ed99aa11b"). InnerVolumeSpecName "kube-api-access-q75zg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:43:09.302572 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.302516 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d738c6a6-4752-45be-b309-227ed99aa11b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d738c6a6-4752-45be-b309-227ed99aa11b" (UID: "d738c6a6-4752-45be-b309-227ed99aa11b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:43:09.343423 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.343391 2565 generic.go:356] "Generic (PLEG): container finished" podID="d738c6a6-4752-45be-b309-227ed99aa11b" containerID="8643171e9c30530e22a9cbbebe6eaea27fe3573d89f393525bb4fa78af5fd138" exitCode=0 Sep 10 17:43:09.343603 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.343432 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84cf7655d-dst5b" event={"ID":"d738c6a6-4752-45be-b309-227ed99aa11b","Type":"ContainerDied","Data":"8643171e9c30530e22a9cbbebe6eaea27fe3573d89f393525bb4fa78af5fd138"} Sep 10 17:43:09.343603 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.343475 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84cf7655d-dst5b" event={"ID":"d738c6a6-4752-45be-b309-227ed99aa11b","Type":"ContainerDied","Data":"7e4c21e9934900ddc97a0ee3166b08e8e52467ca0d50c3276e19b75680738000"} Sep 10 17:43:09.343603 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.343501 2565 scope.go:117] "RemoveContainer" containerID="8643171e9c30530e22a9cbbebe6eaea27fe3573d89f393525bb4fa78af5fd138" Sep 10 17:43:09.343603 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.343435 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84cf7655d-dst5b" Sep 10 17:43:09.352763 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.352735 2565 scope.go:117] "RemoveContainer" containerID="8643171e9c30530e22a9cbbebe6eaea27fe3573d89f393525bb4fa78af5fd138" Sep 10 17:43:09.353137 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:43:09.353112 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8643171e9c30530e22a9cbbebe6eaea27fe3573d89f393525bb4fa78af5fd138\": container with ID starting with 8643171e9c30530e22a9cbbebe6eaea27fe3573d89f393525bb4fa78af5fd138 not found: ID does not exist" containerID="8643171e9c30530e22a9cbbebe6eaea27fe3573d89f393525bb4fa78af5fd138" Sep 10 17:43:09.353216 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.353145 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8643171e9c30530e22a9cbbebe6eaea27fe3573d89f393525bb4fa78af5fd138"} err="failed to get container status \"8643171e9c30530e22a9cbbebe6eaea27fe3573d89f393525bb4fa78af5fd138\": rpc error: code = NotFound desc = could not find container \"8643171e9c30530e22a9cbbebe6eaea27fe3573d89f393525bb4fa78af5fd138\": container with ID starting with 8643171e9c30530e22a9cbbebe6eaea27fe3573d89f393525bb4fa78af5fd138 not found: ID does not exist" Sep 10 17:43:09.381135 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.381098 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84cf7655d-dst5b"] Sep 10 17:43:09.391070 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.391025 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84cf7655d-dst5b"] Sep 10 17:43:09.401383 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.401318 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-oauth-serving-cert\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:43:09.401383 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.401347 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-service-ca\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:43:09.401383 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.401359 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q75zg\" (UniqueName: \"kubernetes.io/projected/d738c6a6-4752-45be-b309-227ed99aa11b-kube-api-access-q75zg\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:43:09.401383 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.401367 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d738c6a6-4752-45be-b309-227ed99aa11b-console-oauth-config\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:43:09.401383 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.401378 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d738c6a6-4752-45be-b309-227ed99aa11b-console-config\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:43:09.401383 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:09.401387 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d738c6a6-4752-45be-b309-227ed99aa11b-console-serving-cert\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:43:10.436345 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:10.436313 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d738c6a6-4752-45be-b309-227ed99aa11b" path="/var/lib/kubelet/pods/d738c6a6-4752-45be-b309-227ed99aa11b/volumes" Sep 10 17:43:17.297019 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:17.296989 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54f8fb7c56-74zxh" Sep 10 17:43:17.551837 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:17.551739 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:17.610424 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:17.610394 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:18.682737 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:18.682702 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:18.743148 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:18.743114 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:24.926928 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:24.926881 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:25.131166 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:25.131130 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:30.151442 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.151409 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5854dbc468-jbfxd"] Sep 10 17:43:30.151922 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.151671 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d738c6a6-4752-45be-b309-227ed99aa11b" containerName="console" Sep 10 17:43:30.151922 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.151683 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d738c6a6-4752-45be-b309-227ed99aa11b" containerName="console" Sep 10 17:43:30.151922 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.151750 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d738c6a6-4752-45be-b309-227ed99aa11b" containerName="console" Sep 10 17:43:30.154178 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.154161 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.156343 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.156322 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 10 17:43:30.157164 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.157150 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-q2dnd\"" Sep 10 17:43:30.163667 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.163639 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 10 17:43:30.163793 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.163704 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 10 17:43:30.163793 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.163739 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 10 17:43:30.163793 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.163748 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 10 17:43:30.168067 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.168044 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 10 17:43:30.176252 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.176228 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5854dbc468-jbfxd"] Sep 10 17:43:30.269233 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.269195 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/452efbb7-5b5e-4b64-8217-cd4e80560ca3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.269233 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.269237 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/452efbb7-5b5e-4b64-8217-cd4e80560ca3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.269482 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.269262 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/452efbb7-5b5e-4b64-8217-cd4e80560ca3-telemeter-client-tls\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.269482 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.269300 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/452efbb7-5b5e-4b64-8217-cd4e80560ca3-metrics-client-ca\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.269482 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.269326 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlbpq\" (UniqueName: \"kubernetes.io/projected/452efbb7-5b5e-4b64-8217-cd4e80560ca3-kube-api-access-tlbpq\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.269482 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.269343 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/452efbb7-5b5e-4b64-8217-cd4e80560ca3-secret-telemeter-client\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.269482 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.269385 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/452efbb7-5b5e-4b64-8217-cd4e80560ca3-serving-certs-ca-bundle\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.269482 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.269404 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/452efbb7-5b5e-4b64-8217-cd4e80560ca3-federate-client-tls\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.370429 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.370391 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/452efbb7-5b5e-4b64-8217-cd4e80560ca3-metrics-client-ca\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.370429 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.370431 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlbpq\" (UniqueName: \"kubernetes.io/projected/452efbb7-5b5e-4b64-8217-cd4e80560ca3-kube-api-access-tlbpq\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.370660 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.370450 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/452efbb7-5b5e-4b64-8217-cd4e80560ca3-secret-telemeter-client\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.370660 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.370567 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/452efbb7-5b5e-4b64-8217-cd4e80560ca3-serving-certs-ca-bundle\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.370660 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.370603 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/452efbb7-5b5e-4b64-8217-cd4e80560ca3-federate-client-tls\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.371172 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.370693 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/452efbb7-5b5e-4b64-8217-cd4e80560ca3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.371172 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.370716 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/452efbb7-5b5e-4b64-8217-cd4e80560ca3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.371172 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.370749 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/452efbb7-5b5e-4b64-8217-cd4e80560ca3-telemeter-client-tls\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.371365 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.371277 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/452efbb7-5b5e-4b64-8217-cd4e80560ca3-metrics-client-ca\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.371365 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.371314 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/452efbb7-5b5e-4b64-8217-cd4e80560ca3-serving-certs-ca-bundle\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.371805 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.371782 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/452efbb7-5b5e-4b64-8217-cd4e80560ca3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.373273 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.373244 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/452efbb7-5b5e-4b64-8217-cd4e80560ca3-secret-telemeter-client\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.373387 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.373315 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/452efbb7-5b5e-4b64-8217-cd4e80560ca3-federate-client-tls\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.373545 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.373525 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/452efbb7-5b5e-4b64-8217-cd4e80560ca3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.373673 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.373653 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/452efbb7-5b5e-4b64-8217-cd4e80560ca3-telemeter-client-tls\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.378591 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.378568 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlbpq\" (UniqueName: \"kubernetes.io/projected/452efbb7-5b5e-4b64-8217-cd4e80560ca3-kube-api-access-tlbpq\") pod \"telemeter-client-5854dbc468-jbfxd\" (UID: \"452efbb7-5b5e-4b64-8217-cd4e80560ca3\") " pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.466519 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.466423 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" Sep 10 17:43:30.739699 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:30.739615 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5854dbc468-jbfxd"] Sep 10 17:43:30.743439 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:43:30.743412 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod452efbb7_5b5e_4b64_8217_cd4e80560ca3.slice/crio-f0607703497ee769f6876006ce3099b4049752afe3fe96b1f340865b16ae0eae WatchSource:0}: Error finding container f0607703497ee769f6876006ce3099b4049752afe3fe96b1f340865b16ae0eae: Status 404 returned error can't find the container with id f0607703497ee769f6876006ce3099b4049752afe3fe96b1f340865b16ae0eae Sep 10 17:43:31.432337 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:31.432306 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" event={"ID":"452efbb7-5b5e-4b64-8217-cd4e80560ca3","Type":"ContainerStarted","Data":"f0607703497ee769f6876006ce3099b4049752afe3fe96b1f340865b16ae0eae"} Sep 10 17:43:32.013282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:32.013229 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:33.444280 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:33.444249 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" event={"ID":"452efbb7-5b5e-4b64-8217-cd4e80560ca3","Type":"ContainerStarted","Data":"8bca3fdb84aa172638c761d34bbf05adfa2e58559475686d3a242675c653573a"} Sep 10 17:43:33.692834 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:33.692805 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:33.798257 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:33.798222 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:34.449727 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.449693 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" event={"ID":"452efbb7-5b5e-4b64-8217-cd4e80560ca3","Type":"ContainerStarted","Data":"081e0e3184b08042bbb32e721413071fc0dbceb9f168e64648f14840ac8d29e4"} Sep 10 17:43:34.450179 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.449747 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" event={"ID":"452efbb7-5b5e-4b64-8217-cd4e80560ca3","Type":"ContainerStarted","Data":"38a6e00d0f4adffb1af59ccc860ccc7cece898090f0c67200d8d7dfe2bbf2f86"} Sep 10 17:43:34.471036 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.470981 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5854dbc468-jbfxd" podStartSLOduration=1.642646565 podStartE2EDuration="4.470961196s" totalImagesPullingTime="2.828314631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:43:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:43:34.470159425 +0000 UTC m=+218.534570155" watchObservedRunningTime="2026-09-10 17:43:34.470961196 +0000 UTC m=+218.535371930" Sep 10 17:43:34.562176 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.562135 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-855ddc5946-jxc4w"] Sep 10 17:43:34.564397 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.564373 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.566560 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.566537 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 17:43:34.566695 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.566589 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9s9rn\"" Sep 10 17:43:34.566695 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.566589 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 17:43:34.567072 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.567053 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 17:43:34.567072 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.567066 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 17:43:34.567072 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.567053 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 17:43:34.573578 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.573554 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 17:43:34.576502 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.576478 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-855ddc5946-jxc4w"] Sep 10 17:43:34.604629 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.604597 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-console-config\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.604629 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.604633 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89720fc8-5353-43e8-9779-fcd4842560f6-console-serving-cert\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.604841 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.604656 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-trusted-ca-bundle\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.604841 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.604727 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-service-ca\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.604841 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.604767 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txz9z\" (UniqueName: \"kubernetes.io/projected/89720fc8-5353-43e8-9779-fcd4842560f6-kube-api-access-txz9z\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.605010 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.604849 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-oauth-serving-cert\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.605010 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.604925 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89720fc8-5353-43e8-9779-fcd4842560f6-console-oauth-config\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.619177 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.619149 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:34.689919 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.689858 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:34.705435 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.705344 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-oauth-serving-cert\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.705435 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.705391 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89720fc8-5353-43e8-9779-fcd4842560f6-console-oauth-config\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.705435 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.705427 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-console-config\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.705688 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.705445 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89720fc8-5353-43e8-9779-fcd4842560f6-console-serving-cert\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.705688 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.705463 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-trusted-ca-bundle\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.705688 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.705480 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-service-ca\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.705688 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.705501 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txz9z\" (UniqueName: \"kubernetes.io/projected/89720fc8-5353-43e8-9779-fcd4842560f6-kube-api-access-txz9z\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.706232 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.706199 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-console-config\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.706323 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.706305 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-service-ca\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.706435 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.706415 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-trusted-ca-bundle\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.706665 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.706650 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-oauth-serving-cert\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.708179 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.708154 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89720fc8-5353-43e8-9779-fcd4842560f6-console-serving-cert\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.708263 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.708155 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89720fc8-5353-43e8-9779-fcd4842560f6-console-oauth-config\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.715460 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.715431 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txz9z\" (UniqueName: \"kubernetes.io/projected/89720fc8-5353-43e8-9779-fcd4842560f6-kube-api-access-txz9z\") pod \"console-855ddc5946-jxc4w\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:34.875331 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:34.875297 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:35.146604 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:35.146571 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-855ddc5946-jxc4w"] Sep 10 17:43:35.149535 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:43:35.149509 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89720fc8_5353_43e8_9779_fcd4842560f6.slice/crio-bd3f4629b83fe72e40a32170002081c77372492e996eeb7263b162201915492d WatchSource:0}: Error finding container bd3f4629b83fe72e40a32170002081c77372492e996eeb7263b162201915492d: Status 404 returned error can't find the container with id bd3f4629b83fe72e40a32170002081c77372492e996eeb7263b162201915492d Sep 10 17:43:35.457806 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:35.457718 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855ddc5946-jxc4w" event={"ID":"89720fc8-5353-43e8-9779-fcd4842560f6","Type":"ContainerStarted","Data":"12b05d77a49a6f3b714364d2cdf230750547c66cebde39eae75e579b849f4604"} Sep 10 17:43:35.457806 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:35.457774 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855ddc5946-jxc4w" event={"ID":"89720fc8-5353-43e8-9779-fcd4842560f6","Type":"ContainerStarted","Data":"bd3f4629b83fe72e40a32170002081c77372492e996eeb7263b162201915492d"} Sep 10 17:43:35.479193 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:35.479143 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-855ddc5946-jxc4w" podStartSLOduration=1.479126185 podStartE2EDuration="1.479126185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:43:35.478863216 +0000 UTC m=+219.543273926" watchObservedRunningTime="2026-09-10 17:43:35.479126185 +0000 UTC m=+219.543536894" Sep 10 17:43:42.989084 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:42.989045 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:44.876026 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:44.875976 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:44.876431 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:44.876047 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:44.881105 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:44.881077 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:45.499768 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:45.499734 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:43:45.671701 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:45.671651 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:45.773203 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:45.773116 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:43:45.844565 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:43:45.844532 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:26.573569 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:44:26.573487 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:26.644207 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:44:26.644177 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:40.030618 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:44:40.030580 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:40.091559 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:44:40.091528 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:52.033420 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:44:52.033378 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:52.132673 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:44:52.132647 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:53.958413 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:44:53.958379 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:54.013048 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:44:54.013016 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:44:56.343444 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:44:56.343410 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g2pbs_f558fde8-d070-4538-bb97-19bb59832b85/ovn-acl-logging/0.log" Sep 10 17:44:56.348415 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:44:56.348385 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g2pbs_f558fde8-d070-4538-bb97-19bb59832b85/ovn-acl-logging/0.log" Sep 10 17:44:56.349603 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:44:56.349585 2565 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 17:45:03.712606 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:03.712571 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:03.810523 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:03.810493 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:04.358596 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:04.358565 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-855ddc5946-jxc4w"] Sep 10 17:45:04.460558 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:04.460519 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:04.568406 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:04.568359 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:04.629487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:04.629410 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:28.753463 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:28.753419 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:29.384048 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.384010 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-855ddc5946-jxc4w" podUID="89720fc8-5353-43e8-9779-fcd4842560f6" containerName="console" containerID="cri-o://12b05d77a49a6f3b714364d2cdf230750547c66cebde39eae75e579b849f4604" gracePeriod=15 Sep 10 17:45:29.614056 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.614032 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:45:29.682769 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.682684 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-trusted-ca-bundle\") pod \"89720fc8-5353-43e8-9779-fcd4842560f6\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " Sep 10 17:45:29.682769 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.682745 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txz9z\" (UniqueName: \"kubernetes.io/projected/89720fc8-5353-43e8-9779-fcd4842560f6-kube-api-access-txz9z\") pod \"89720fc8-5353-43e8-9779-fcd4842560f6\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " Sep 10 17:45:29.682769 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.682767 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-service-ca\") pod \"89720fc8-5353-43e8-9779-fcd4842560f6\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " Sep 10 17:45:29.683053 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.682791 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-oauth-serving-cert\") pod \"89720fc8-5353-43e8-9779-fcd4842560f6\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " Sep 10 17:45:29.683053 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.682825 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-console-config\") pod \"89720fc8-5353-43e8-9779-fcd4842560f6\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " Sep 10 17:45:29.683053 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.682952 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89720fc8-5353-43e8-9779-fcd4842560f6-console-serving-cert\") pod \"89720fc8-5353-43e8-9779-fcd4842560f6\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " Sep 10 17:45:29.683053 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.682998 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89720fc8-5353-43e8-9779-fcd4842560f6-console-oauth-config\") pod \"89720fc8-5353-43e8-9779-fcd4842560f6\" (UID: \"89720fc8-5353-43e8-9779-fcd4842560f6\") " Sep 10 17:45:29.683236 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.683211 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "89720fc8-5353-43e8-9779-fcd4842560f6" (UID: "89720fc8-5353-43e8-9779-fcd4842560f6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:45:29.683283 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.683236 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-console-config" (OuterVolumeSpecName: "console-config") pod "89720fc8-5353-43e8-9779-fcd4842560f6" (UID: "89720fc8-5353-43e8-9779-fcd4842560f6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:45:29.683283 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.683243 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-service-ca" (OuterVolumeSpecName: "service-ca") pod "89720fc8-5353-43e8-9779-fcd4842560f6" (UID: "89720fc8-5353-43e8-9779-fcd4842560f6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:45:29.683516 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.683490 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "89720fc8-5353-43e8-9779-fcd4842560f6" (UID: "89720fc8-5353-43e8-9779-fcd4842560f6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:45:29.685137 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.685114 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89720fc8-5353-43e8-9779-fcd4842560f6-kube-api-access-txz9z" (OuterVolumeSpecName: "kube-api-access-txz9z") pod "89720fc8-5353-43e8-9779-fcd4842560f6" (UID: "89720fc8-5353-43e8-9779-fcd4842560f6"). InnerVolumeSpecName "kube-api-access-txz9z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:45:29.685247 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.685113 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89720fc8-5353-43e8-9779-fcd4842560f6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "89720fc8-5353-43e8-9779-fcd4842560f6" (UID: "89720fc8-5353-43e8-9779-fcd4842560f6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:45:29.685247 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.685173 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89720fc8-5353-43e8-9779-fcd4842560f6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "89720fc8-5353-43e8-9779-fcd4842560f6" (UID: "89720fc8-5353-43e8-9779-fcd4842560f6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:45:29.783754 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.783715 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-console-config\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:45:29.783754 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.783743 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89720fc8-5353-43e8-9779-fcd4842560f6-console-serving-cert\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:45:29.783754 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.783755 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89720fc8-5353-43e8-9779-fcd4842560f6-console-oauth-config\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:45:29.783754 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.783765 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-trusted-ca-bundle\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:45:29.784252 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.783774 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txz9z\" (UniqueName: \"kubernetes.io/projected/89720fc8-5353-43e8-9779-fcd4842560f6-kube-api-access-txz9z\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:45:29.784252 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.783782 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-service-ca\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:45:29.784252 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.783791 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89720fc8-5353-43e8-9779-fcd4842560f6-oauth-serving-cert\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:45:29.882432 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.882393 2565 generic.go:356] "Generic (PLEG): container finished" podID="89720fc8-5353-43e8-9779-fcd4842560f6" containerID="12b05d77a49a6f3b714364d2cdf230750547c66cebde39eae75e579b849f4604" exitCode=0 Sep 10 17:45:29.882611 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.882447 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855ddc5946-jxc4w" Sep 10 17:45:29.882611 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.882483 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855ddc5946-jxc4w" event={"ID":"89720fc8-5353-43e8-9779-fcd4842560f6","Type":"ContainerDied","Data":"12b05d77a49a6f3b714364d2cdf230750547c66cebde39eae75e579b849f4604"} Sep 10 17:45:29.882611 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.882536 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855ddc5946-jxc4w" event={"ID":"89720fc8-5353-43e8-9779-fcd4842560f6","Type":"ContainerDied","Data":"bd3f4629b83fe72e40a32170002081c77372492e996eeb7263b162201915492d"} Sep 10 17:45:29.882611 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.882556 2565 scope.go:117] "RemoveContainer" containerID="12b05d77a49a6f3b714364d2cdf230750547c66cebde39eae75e579b849f4604" Sep 10 17:45:29.891218 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.891197 2565 scope.go:117] "RemoveContainer" containerID="12b05d77a49a6f3b714364d2cdf230750547c66cebde39eae75e579b849f4604" Sep 10 17:45:29.891490 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:45:29.891474 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12b05d77a49a6f3b714364d2cdf230750547c66cebde39eae75e579b849f4604\": container with ID starting with 12b05d77a49a6f3b714364d2cdf230750547c66cebde39eae75e579b849f4604 not found: ID does not exist" containerID="12b05d77a49a6f3b714364d2cdf230750547c66cebde39eae75e579b849f4604" Sep 10 17:45:29.891533 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.891496 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12b05d77a49a6f3b714364d2cdf230750547c66cebde39eae75e579b849f4604"} err="failed to get container status \"12b05d77a49a6f3b714364d2cdf230750547c66cebde39eae75e579b849f4604\": rpc error: code = NotFound desc = could not find container \"12b05d77a49a6f3b714364d2cdf230750547c66cebde39eae75e579b849f4604\": container with ID starting with 12b05d77a49a6f3b714364d2cdf230750547c66cebde39eae75e579b849f4604 not found: ID does not exist" Sep 10 17:45:29.902925 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.902879 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-855ddc5946-jxc4w"] Sep 10 17:45:29.907693 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:29.907669 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-855ddc5946-jxc4w"] Sep 10 17:45:30.436976 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:30.436943 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89720fc8-5353-43e8-9779-fcd4842560f6" path="/var/lib/kubelet/pods/89720fc8-5353-43e8-9779-fcd4842560f6/volumes" Sep 10 17:45:41.291994 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:41.291956 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:45:41.393660 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:45:41.393622 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:03.001499 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:03.001464 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:04.440904 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:04.440838 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:06.590779 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:06.590742 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:06.653069 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:06.653034 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:09.251714 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.251678 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:09.352712 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.352679 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:09.408243 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.408208 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8s5kc"] Sep 10 17:46:09.408488 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.408476 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89720fc8-5353-43e8-9779-fcd4842560f6" containerName="console" Sep 10 17:46:09.408530 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.408489 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="89720fc8-5353-43e8-9779-fcd4842560f6" containerName="console" Sep 10 17:46:09.408568 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.408552 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="89720fc8-5353-43e8-9779-fcd4842560f6" containerName="console" Sep 10 17:46:09.412550 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.412534 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8s5kc" Sep 10 17:46:09.414548 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.414525 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 10 17:46:09.414647 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.414533 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 10 17:46:09.415011 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.414995 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-cczjn\"" Sep 10 17:46:09.421018 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.420998 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8s5kc"] Sep 10 17:46:09.588705 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.588620 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dshwm\" (UniqueName: \"kubernetes.io/projected/9aaa82d7-82d2-4f4c-8bf6-a8ace3ba17de-kube-api-access-dshwm\") pod \"cert-manager-webhook-6486ffd997-8s5kc\" (UID: \"9aaa82d7-82d2-4f4c-8bf6-a8ace3ba17de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8s5kc" Sep 10 17:46:09.588705 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.588682 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9aaa82d7-82d2-4f4c-8bf6-a8ace3ba17de-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8s5kc\" (UID: \"9aaa82d7-82d2-4f4c-8bf6-a8ace3ba17de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8s5kc" Sep 10 17:46:09.690039 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.690008 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dshwm\" (UniqueName: \"kubernetes.io/projected/9aaa82d7-82d2-4f4c-8bf6-a8ace3ba17de-kube-api-access-dshwm\") pod \"cert-manager-webhook-6486ffd997-8s5kc\" (UID: \"9aaa82d7-82d2-4f4c-8bf6-a8ace3ba17de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8s5kc" Sep 10 17:46:09.690195 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.690060 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9aaa82d7-82d2-4f4c-8bf6-a8ace3ba17de-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8s5kc\" (UID: \"9aaa82d7-82d2-4f4c-8bf6-a8ace3ba17de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8s5kc" Sep 10 17:46:09.698187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.698158 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9aaa82d7-82d2-4f4c-8bf6-a8ace3ba17de-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8s5kc\" (UID: \"9aaa82d7-82d2-4f4c-8bf6-a8ace3ba17de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8s5kc" Sep 10 17:46:09.698480 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.698460 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dshwm\" (UniqueName: \"kubernetes.io/projected/9aaa82d7-82d2-4f4c-8bf6-a8ace3ba17de-kube-api-access-dshwm\") pod \"cert-manager-webhook-6486ffd997-8s5kc\" (UID: \"9aaa82d7-82d2-4f4c-8bf6-a8ace3ba17de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8s5kc" Sep 10 17:46:09.722307 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.722284 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8s5kc" Sep 10 17:46:09.995470 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.995439 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8s5kc"] Sep 10 17:46:09.998617 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:09.998590 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:09.998770 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:46:09.998748 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9aaa82d7_82d2_4f4c_8bf6_a8ace3ba17de.slice/crio-af21cf9e3049659216066916c099e44b897fa7f7be4122708511225bf8ac21cc WatchSource:0}: Error finding container af21cf9e3049659216066916c099e44b897fa7f7be4122708511225bf8ac21cc: Status 404 returned error can't find the container with id af21cf9e3049659216066916c099e44b897fa7f7be4122708511225bf8ac21cc Sep 10 17:46:10.000739 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:10.000724 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 17:46:10.033032 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:10.033008 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8s5kc" event={"ID":"9aaa82d7-82d2-4f4c-8bf6-a8ace3ba17de","Type":"ContainerStarted","Data":"af21cf9e3049659216066916c099e44b897fa7f7be4122708511225bf8ac21cc"} Sep 10 17:46:10.145117 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:10.145088 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:11.277191 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:11.277123 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:11.688937 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:11.688845 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:11.743501 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:11.743461 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dvtvx"] Sep 10 17:46:11.752664 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:11.752638 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dvtvx" Sep 10 17:46:11.756364 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:11.755207 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-blhn7\"" Sep 10 17:46:11.756613 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:11.756577 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dvtvx"] Sep 10 17:46:11.807752 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:11.807710 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99380c52-79ce-41df-932f-0431aa925286-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dvtvx\" (UID: \"99380c52-79ce-41df-932f-0431aa925286\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dvtvx" Sep 10 17:46:11.807946 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:11.807756 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvddw\" (UniqueName: \"kubernetes.io/projected/99380c52-79ce-41df-932f-0431aa925286-kube-api-access-gvddw\") pod \"cert-manager-cainjector-559cdd8c87-dvtvx\" (UID: \"99380c52-79ce-41df-932f-0431aa925286\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dvtvx" Sep 10 17:46:11.908626 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:11.908589 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99380c52-79ce-41df-932f-0431aa925286-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dvtvx\" (UID: \"99380c52-79ce-41df-932f-0431aa925286\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dvtvx" Sep 10 17:46:11.908626 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:11.908632 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvddw\" (UniqueName: \"kubernetes.io/projected/99380c52-79ce-41df-932f-0431aa925286-kube-api-access-gvddw\") pod \"cert-manager-cainjector-559cdd8c87-dvtvx\" (UID: \"99380c52-79ce-41df-932f-0431aa925286\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dvtvx" Sep 10 17:46:11.917873 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:11.917843 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99380c52-79ce-41df-932f-0431aa925286-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dvtvx\" (UID: \"99380c52-79ce-41df-932f-0431aa925286\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dvtvx" Sep 10 17:46:11.918280 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:11.918256 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvddw\" (UniqueName: \"kubernetes.io/projected/99380c52-79ce-41df-932f-0431aa925286-kube-api-access-gvddw\") pod \"cert-manager-cainjector-559cdd8c87-dvtvx\" (UID: \"99380c52-79ce-41df-932f-0431aa925286\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dvtvx" Sep 10 17:46:12.065967 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:12.065877 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dvtvx" Sep 10 17:46:12.792361 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:12.792329 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:12.851118 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:12.851074 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:13.227166 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:13.227138 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dvtvx"] Sep 10 17:46:13.229416 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:46:13.229389 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99380c52_79ce_41df_932f_0431aa925286.slice/crio-8824158a15353bcc7a538673fd9d7337af4f431137d14cd971d5bb6c06a8f9dc WatchSource:0}: Error finding container 8824158a15353bcc7a538673fd9d7337af4f431137d14cd971d5bb6c06a8f9dc: Status 404 returned error can't find the container with id 8824158a15353bcc7a538673fd9d7337af4f431137d14cd971d5bb6c06a8f9dc Sep 10 17:46:14.052045 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:14.052004 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8s5kc" event={"ID":"9aaa82d7-82d2-4f4c-8bf6-a8ace3ba17de","Type":"ContainerStarted","Data":"558ceb5af7acbf1473b2b729edfeb5bd69e60dcbc49206f906dc279cbf066672"} Sep 10 17:46:14.052488 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:14.052055 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8s5kc" Sep 10 17:46:14.053562 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:14.053534 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dvtvx" event={"ID":"99380c52-79ce-41df-932f-0431aa925286","Type":"ContainerStarted","Data":"f863daf456a1f9d17746580f6464a85f89a10fc28882ed16359f62d61c6771f8"} Sep 10 17:46:14.053669 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:14.053569 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dvtvx" event={"ID":"99380c52-79ce-41df-932f-0431aa925286","Type":"ContainerStarted","Data":"8824158a15353bcc7a538673fd9d7337af4f431137d14cd971d5bb6c06a8f9dc"} Sep 10 17:46:14.070950 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:14.070904 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-8s5kc" podStartSLOduration=2.041098242 podStartE2EDuration="5.070875582s" totalImagesPullingTime="3.02977734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:46:14.0703322 +0000 UTC m=+378.134742927" watchObservedRunningTime="2026-09-10 17:46:14.070875582 +0000 UTC m=+378.135286292" Sep 10 17:46:14.085589 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:14.085548 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dvtvx" podStartSLOduration=3.085534618 podStartE2EDuration="3.085534618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:46:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:46:14.084173225 +0000 UTC m=+378.148583937" watchObservedRunningTime="2026-09-10 17:46:14.085534618 +0000 UTC m=+378.149945327" Sep 10 17:46:17.358563 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:17.358531 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:17.497943 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:17.497911 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:20.059875 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:20.059843 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8s5kc" Sep 10 17:46:20.450284 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:20.450245 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:20.600645 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:20.600615 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:20.700803 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:20.700716 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:28.413387 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:28.413352 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:28.517283 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:28.517250 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:30.590357 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:30.590327 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:44.652040 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:44.652005 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:44.752495 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:44.752462 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:46.268505 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:46.268477 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:46.356838 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:46.356810 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:47.057598 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.057562 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:47.509063 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.509007 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:47.618254 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.618224 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:47.651982 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.651952 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4"] Sep 10 17:46:47.656185 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.656164 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.659767 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.659740 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 17:46:47.660084 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.660060 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 17:46:47.660192 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.660105 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 17:46:47.660192 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.660112 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 17:46:47.660486 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.660472 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qkbhg\"" Sep 10 17:46:47.679682 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.679658 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4"] Sep 10 17:46:47.798423 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.798340 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-manifests\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.798423 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.798378 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.798600 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.798455 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-tmp\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.798600 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.798481 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c5r4\" (UniqueName: \"kubernetes.io/projected/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-kube-api-access-4c5r4\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.798600 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.798522 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.814186 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.814158 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:47.894205 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.894177 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:47.898865 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.898830 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-tmp\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.898865 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.898876 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c5r4\" (UniqueName: \"kubernetes.io/projected/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-kube-api-access-4c5r4\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.899058 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.898924 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.899058 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.898975 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-manifests\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.899058 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.899003 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.899424 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.899371 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-manifests\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.901218 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.901191 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-tmp\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.901375 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.901355 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.901439 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.901405 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.911050 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.911030 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c5r4\" (UniqueName: \"kubernetes.io/projected/60fc9b3c-331c-4a37-ab1a-21fe46eb0066-kube-api-access-4c5r4\") pod \"opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4\" (UID: \"60fc9b3c-331c-4a37-ab1a-21fe46eb0066\") " pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.966002 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.965975 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:47.991978 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:47.991947 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:48.119450 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:48.119371 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:48.235225 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:48.235193 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:48.269398 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:48.269370 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4"] Sep 10 17:46:48.274192 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:46:48.274169 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60fc9b3c_331c_4a37_ab1a_21fe46eb0066.slice/crio-a13c334ae5e54180f78ffd4a796ae0477d0b12b8c858b0f61412de415600468d WatchSource:0}: Error finding container a13c334ae5e54180f78ffd4a796ae0477d0b12b8c858b0f61412de415600468d: Status 404 returned error can't find the container with id a13c334ae5e54180f78ffd4a796ae0477d0b12b8c858b0f61412de415600468d Sep 10 17:46:48.366556 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:48.366524 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:49.188556 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:49.188482 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" event={"ID":"60fc9b3c-331c-4a37-ab1a-21fe46eb0066","Type":"ContainerStarted","Data":"a13c334ae5e54180f78ffd4a796ae0477d0b12b8c858b0f61412de415600468d"} Sep 10 17:46:51.198804 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:51.198768 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" event={"ID":"60fc9b3c-331c-4a37-ab1a-21fe46eb0066","Type":"ContainerStarted","Data":"985e8cdacfc3cd368329a279e88db111b553c87c46a24cccbc1da7464c2d3ce9"} Sep 10 17:46:52.203930 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:52.203897 2565 generic.go:356] "Generic (PLEG): container finished" podID="60fc9b3c-331c-4a37-ab1a-21fe46eb0066" containerID="985e8cdacfc3cd368329a279e88db111b553c87c46a24cccbc1da7464c2d3ce9" exitCode=0 Sep 10 17:46:52.204278 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:52.203950 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" event={"ID":"60fc9b3c-331c-4a37-ab1a-21fe46eb0066","Type":"ContainerDied","Data":"985e8cdacfc3cd368329a279e88db111b553c87c46a24cccbc1da7464c2d3ce9"} Sep 10 17:46:53.209864 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:53.209825 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" event={"ID":"60fc9b3c-331c-4a37-ab1a-21fe46eb0066","Type":"ContainerStarted","Data":"ff19b868bebc738d003d5d792c0270a3df22261f50ecb150f4103694b5071e8d"} Sep 10 17:46:53.210314 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:53.210013 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:46:53.239774 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:53.239731 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" podStartSLOduration=2.072506857 podStartE2EDuration="6.23971157s" totalImagesPullingTime="3.122590738s" totalInitContainerRuntime="1.044613975s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:46:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:46:53.239085182 +0000 UTC m=+417.303495913" watchObservedRunningTime="2026-09-10 17:46:53.23971157 +0000 UTC m=+417.304122280" Sep 10 17:46:59.159938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:59.159879 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:59.302686 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:59.302653 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:59.445823 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:59.445737 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:46:59.536475 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:46:59.536438 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:04.216450 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:04.216420 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4" Sep 10 17:47:06.634550 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:06.634516 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:06.779172 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:06.779141 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:07.167924 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.167865 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:07.379520 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.379490 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p"] Sep 10 17:47:07.389615 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.389592 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p" Sep 10 17:47:07.394739 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.394721 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 17:47:07.395034 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.395007 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xjznp\"" Sep 10 17:47:07.395034 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.395010 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 10 17:47:07.395204 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.395069 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 17:47:07.395378 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.395363 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 10 17:47:07.400695 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.400673 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p"] Sep 10 17:47:07.448837 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.448771 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:07.563942 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.563907 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6335b441-2450-4b00-b46a-c91a2101a6c5-tmp\") pod \"kube-auth-proxy-6f88844dfb-mp78p\" (UID: \"6335b441-2450-4b00-b46a-c91a2101a6c5\") " pod="openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p" Sep 10 17:47:07.564154 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.563959 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6335b441-2450-4b00-b46a-c91a2101a6c5-tls-certs\") pod \"kube-auth-proxy-6f88844dfb-mp78p\" (UID: \"6335b441-2450-4b00-b46a-c91a2101a6c5\") " pod="openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p" Sep 10 17:47:07.564154 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.564001 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfl6t\" (UniqueName: \"kubernetes.io/projected/6335b441-2450-4b00-b46a-c91a2101a6c5-kube-api-access-cfl6t\") pod \"kube-auth-proxy-6f88844dfb-mp78p\" (UID: \"6335b441-2450-4b00-b46a-c91a2101a6c5\") " pod="openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p" Sep 10 17:47:07.664954 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.664918 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6335b441-2450-4b00-b46a-c91a2101a6c5-tmp\") pod \"kube-auth-proxy-6f88844dfb-mp78p\" (UID: \"6335b441-2450-4b00-b46a-c91a2101a6c5\") " pod="openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p" Sep 10 17:47:07.664954 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.664967 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6335b441-2450-4b00-b46a-c91a2101a6c5-tls-certs\") pod \"kube-auth-proxy-6f88844dfb-mp78p\" (UID: \"6335b441-2450-4b00-b46a-c91a2101a6c5\") " pod="openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p" Sep 10 17:47:07.665467 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.665015 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfl6t\" (UniqueName: \"kubernetes.io/projected/6335b441-2450-4b00-b46a-c91a2101a6c5-kube-api-access-cfl6t\") pod \"kube-auth-proxy-6f88844dfb-mp78p\" (UID: \"6335b441-2450-4b00-b46a-c91a2101a6c5\") " pod="openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p" Sep 10 17:47:07.667231 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.667200 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6335b441-2450-4b00-b46a-c91a2101a6c5-tmp\") pod \"kube-auth-proxy-6f88844dfb-mp78p\" (UID: \"6335b441-2450-4b00-b46a-c91a2101a6c5\") " pod="openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p" Sep 10 17:47:07.667416 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.667399 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6335b441-2450-4b00-b46a-c91a2101a6c5-tls-certs\") pod \"kube-auth-proxy-6f88844dfb-mp78p\" (UID: \"6335b441-2450-4b00-b46a-c91a2101a6c5\") " pod="openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p" Sep 10 17:47:07.678314 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.678279 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfl6t\" (UniqueName: \"kubernetes.io/projected/6335b441-2450-4b00-b46a-c91a2101a6c5-kube-api-access-cfl6t\") pod \"kube-auth-proxy-6f88844dfb-mp78p\" (UID: \"6335b441-2450-4b00-b46a-c91a2101a6c5\") " pod="openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p" Sep 10 17:47:07.700191 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.700129 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p" Sep 10 17:47:07.977254 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:07.977233 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p"] Sep 10 17:47:07.979591 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:47:07.979561 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6335b441_2450_4b00_b46a_c91a2101a6c5.slice/crio-cddd8adc6b315e7c2c6a93a810e159d8200a01c716cc81625bf04270935ae87b WatchSource:0}: Error finding container cddd8adc6b315e7c2c6a93a810e159d8200a01c716cc81625bf04270935ae87b: Status 404 returned error can't find the container with id cddd8adc6b315e7c2c6a93a810e159d8200a01c716cc81625bf04270935ae87b Sep 10 17:47:08.271353 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:08.271265 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p" event={"ID":"6335b441-2450-4b00-b46a-c91a2101a6c5","Type":"ContainerStarted","Data":"cddd8adc6b315e7c2c6a93a810e159d8200a01c716cc81625bf04270935ae87b"} Sep 10 17:47:09.939523 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:09.939489 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:10.069670 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.069636 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g"] Sep 10 17:47:10.073597 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.073574 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.080611 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.080578 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 10 17:47:10.080733 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.080698 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7r2dp\"" Sep 10 17:47:10.082022 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.081803 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 10 17:47:10.082022 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.081834 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 10 17:47:10.086714 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.086678 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 10 17:47:10.087101 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.087076 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g"] Sep 10 17:47:10.187703 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.187663 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.187919 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.187730 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.187919 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.187833 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.187919 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.187868 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.188078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.187972 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.188078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.188005 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p28bf\" (UniqueName: \"kubernetes.io/projected/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-kube-api-access-p28bf\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.188078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.188029 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.208180 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.208105 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:10.289276 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.289241 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.289455 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.289309 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.289455 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.289345 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.289455 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.289388 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.289455 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.289412 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.289658 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.289460 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.289658 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.289488 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p28bf\" (UniqueName: \"kubernetes.io/projected/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-kube-api-access-p28bf\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.290235 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.290204 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.292134 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.292110 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.292281 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.292253 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.292353 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.292307 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.292428 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.292400 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.312491 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.312466 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p28bf\" (UniqueName: \"kubernetes.io/projected/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-kube-api-access-p28bf\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.317686 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.317644 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r4v2g\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.378492 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.378460 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:10.385198 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.385174 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:10.829285 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:10.829178 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g"] Sep 10 17:47:10.832146 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:47:10.832119 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfd9dada_0f13_4a89_9a3e_1ae69f85d0ce.slice/crio-55331d0cd3d630e9cb6efc01e7f16813238f420b8bd01b5abce78a96acb35cf5 WatchSource:0}: Error finding container 55331d0cd3d630e9cb6efc01e7f16813238f420b8bd01b5abce78a96acb35cf5: Status 404 returned error can't find the container with id 55331d0cd3d630e9cb6efc01e7f16813238f420b8bd01b5abce78a96acb35cf5 Sep 10 17:47:11.289077 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:11.289035 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" event={"ID":"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce","Type":"ContainerStarted","Data":"55331d0cd3d630e9cb6efc01e7f16813238f420b8bd01b5abce78a96acb35cf5"} Sep 10 17:47:11.291446 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:11.291410 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p" event={"ID":"6335b441-2450-4b00-b46a-c91a2101a6c5","Type":"ContainerStarted","Data":"0a7cf2c0d2a706a0cd7b585434a094f274bfb288e8a32b8cc96f95345a878ffe"} Sep 10 17:47:11.320086 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:11.320016 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6f88844dfb-mp78p" podStartSLOduration=1.70400282 podStartE2EDuration="4.319997594s" totalImagesPullingTime="2.615994774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:47:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:47:11.318222338 +0000 UTC m=+435.382633073" watchObservedRunningTime="2026-09-10 17:47:11.319997594 +0000 UTC m=+435.384408306" Sep 10 17:47:12.308722 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:12.308686 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:12.978161 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:12.978127 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 17:47:12.978232 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:12.978192 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 17:47:13.301978 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:13.301878 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" event={"ID":"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce","Type":"ContainerStarted","Data":"98f91a3795cc47ac1cb4e672b0eccf2bcdfbf835e32ff2a6d233a70eb33f29db"} Sep 10 17:47:13.302135 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:13.302076 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:13.322597 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:13.322552 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" podStartSLOduration=1.179316255 podStartE2EDuration="3.322539689s" totalImagesPullingTime="2.143223434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:47:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:47:13.32175936 +0000 UTC m=+437.386170096" watchObservedRunningTime="2026-09-10 17:47:13.322539689 +0000 UTC m=+437.386950399" Sep 10 17:47:13.865682 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:13.865645 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:13.965512 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:13.965479 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:14.215450 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:14.215419 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:14.308211 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:14.308172 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-r4v2g container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 17:47:14.308383 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:14.308228 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" podUID="dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 17:47:14.320557 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:14.320533 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:14.422934 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:14.422902 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:17.307457 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:17.307425 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:17.650013 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:17.649980 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:17.750671 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:17.750636 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:17.904317 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:17.904245 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:18.190092 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:18.190016 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:18.287634 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:18.287602 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:27.477524 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.477446 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:27.551550 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.551513 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8"] Sep 10 17:47:27.555187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.555161 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8" Sep 10 17:47:27.558240 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.558217 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 10 17:47:27.558531 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.558515 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-dld5v\"" Sep 10 17:47:27.566265 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.566238 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8"] Sep 10 17:47:27.569571 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.569550 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8"] Sep 10 17:47:27.569669 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.569639 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8" Sep 10 17:47:27.573099 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.573071 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Sep 10 17:47:27.573236 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.573117 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-vj7zf\"" Sep 10 17:47:27.584605 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.584578 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8"] Sep 10 17:47:27.588264 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.588239 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:27.630302 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.630263 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn5hs\" (UniqueName: \"kubernetes.io/projected/f179b291-78a4-416a-9950-d04c61a061fc-kube-api-access-xn5hs\") pod \"llmisvc-controller-manager-777c6bd7d5-jj5r8\" (UID: \"f179b291-78a4-416a-9950-d04c61a061fc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8" Sep 10 17:47:27.630481 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.630312 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7915f0f9-6e25-48a0-b177-0b406b6c6d87-cert\") pod \"kserve-controller-manager-d6f4b65b8-4wwx8\" (UID: \"7915f0f9-6e25-48a0-b177-0b406b6c6d87\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8" Sep 10 17:47:27.630481 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.630330 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f179b291-78a4-416a-9950-d04c61a061fc-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jj5r8\" (UID: \"f179b291-78a4-416a-9950-d04c61a061fc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8" Sep 10 17:47:27.630481 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.630383 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz8td\" (UniqueName: \"kubernetes.io/projected/7915f0f9-6e25-48a0-b177-0b406b6c6d87-kube-api-access-lz8td\") pod \"kserve-controller-manager-d6f4b65b8-4wwx8\" (UID: \"7915f0f9-6e25-48a0-b177-0b406b6c6d87\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8" Sep 10 17:47:27.662336 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.662302 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:27.731355 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.731250 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7915f0f9-6e25-48a0-b177-0b406b6c6d87-cert\") pod \"kserve-controller-manager-d6f4b65b8-4wwx8\" (UID: \"7915f0f9-6e25-48a0-b177-0b406b6c6d87\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8" Sep 10 17:47:27.731355 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.731297 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f179b291-78a4-416a-9950-d04c61a061fc-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jj5r8\" (UID: \"f179b291-78a4-416a-9950-d04c61a061fc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8" Sep 10 17:47:27.731595 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.731426 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz8td\" (UniqueName: \"kubernetes.io/projected/7915f0f9-6e25-48a0-b177-0b406b6c6d87-kube-api-access-lz8td\") pod \"kserve-controller-manager-d6f4b65b8-4wwx8\" (UID: \"7915f0f9-6e25-48a0-b177-0b406b6c6d87\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8" Sep 10 17:47:27.731595 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.731557 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn5hs\" (UniqueName: \"kubernetes.io/projected/f179b291-78a4-416a-9950-d04c61a061fc-kube-api-access-xn5hs\") pod \"llmisvc-controller-manager-777c6bd7d5-jj5r8\" (UID: \"f179b291-78a4-416a-9950-d04c61a061fc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8" Sep 10 17:47:27.734033 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.734010 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7915f0f9-6e25-48a0-b177-0b406b6c6d87-cert\") pod \"kserve-controller-manager-d6f4b65b8-4wwx8\" (UID: \"7915f0f9-6e25-48a0-b177-0b406b6c6d87\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8" Sep 10 17:47:27.734274 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.734252 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f179b291-78a4-416a-9950-d04c61a061fc-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jj5r8\" (UID: \"f179b291-78a4-416a-9950-d04c61a061fc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8" Sep 10 17:47:27.753577 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.753541 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz8td\" (UniqueName: \"kubernetes.io/projected/7915f0f9-6e25-48a0-b177-0b406b6c6d87-kube-api-access-lz8td\") pod \"kserve-controller-manager-d6f4b65b8-4wwx8\" (UID: \"7915f0f9-6e25-48a0-b177-0b406b6c6d87\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8" Sep 10 17:47:27.753707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.753683 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn5hs\" (UniqueName: \"kubernetes.io/projected/f179b291-78a4-416a-9950-d04c61a061fc-kube-api-access-xn5hs\") pod \"llmisvc-controller-manager-777c6bd7d5-jj5r8\" (UID: \"f179b291-78a4-416a-9950-d04c61a061fc\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8" Sep 10 17:47:27.776952 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.776923 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:27.866724 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.866685 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8" Sep 10 17:47:27.880907 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:27.880861 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8" Sep 10 17:47:28.110869 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.110777 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g"] Sep 10 17:47:28.111043 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.111011 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" podUID="dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce" containerName="discovery" containerID="cri-o://98f91a3795cc47ac1cb4e672b0eccf2bcdfbf835e32ff2a6d233a70eb33f29db" gracePeriod=30 Sep 10 17:47:28.143174 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.143150 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8"] Sep 10 17:47:28.159096 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.159072 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8"] Sep 10 17:47:28.168840 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:47:28.168813 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7915f0f9_6e25_48a0_b177_0b406b6c6d87.slice/crio-42c174a170fd29a038e858d1ce8c005a1bd106b9d8e82d2f72e6b5a99b451823 WatchSource:0}: Error finding container 42c174a170fd29a038e858d1ce8c005a1bd106b9d8e82d2f72e6b5a99b451823: Status 404 returned error can't find the container with id 42c174a170fd29a038e858d1ce8c005a1bd106b9d8e82d2f72e6b5a99b451823 Sep 10 17:47:28.185758 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:47:28.185725 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf179b291_78a4_416a_9950_d04c61a061fc.slice/crio-142c06d2471ef16d8c331d9543d5053bf374e3bad93ce57b8556b25fbb15a085 WatchSource:0}: Error finding container 142c06d2471ef16d8c331d9543d5053bf374e3bad93ce57b8556b25fbb15a085: Status 404 returned error can't find the container with id 142c06d2471ef16d8c331d9543d5053bf374e3bad93ce57b8556b25fbb15a085 Sep 10 17:47:28.344408 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.344381 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:28.371081 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.370987 2565 generic.go:356] "Generic (PLEG): container finished" podID="dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce" containerID="98f91a3795cc47ac1cb4e672b0eccf2bcdfbf835e32ff2a6d233a70eb33f29db" exitCode=0 Sep 10 17:47:28.371081 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.371043 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" Sep 10 17:47:28.371263 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.371070 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" event={"ID":"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce","Type":"ContainerDied","Data":"98f91a3795cc47ac1cb4e672b0eccf2bcdfbf835e32ff2a6d233a70eb33f29db"} Sep 10 17:47:28.371263 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.371168 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g" event={"ID":"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce","Type":"ContainerDied","Data":"55331d0cd3d630e9cb6efc01e7f16813238f420b8bd01b5abce78a96acb35cf5"} Sep 10 17:47:28.371263 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.371202 2565 scope.go:117] "RemoveContainer" containerID="98f91a3795cc47ac1cb4e672b0eccf2bcdfbf835e32ff2a6d233a70eb33f29db" Sep 10 17:47:28.372743 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.372722 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8" event={"ID":"f179b291-78a4-416a-9950-d04c61a061fc","Type":"ContainerStarted","Data":"142c06d2471ef16d8c331d9543d5053bf374e3bad93ce57b8556b25fbb15a085"} Sep 10 17:47:28.374386 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.374361 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8" event={"ID":"7915f0f9-6e25-48a0-b177-0b406b6c6d87","Type":"ContainerStarted","Data":"42c174a170fd29a038e858d1ce8c005a1bd106b9d8e82d2f72e6b5a99b451823"} Sep 10 17:47:28.383351 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.383330 2565 scope.go:117] "RemoveContainer" containerID="98f91a3795cc47ac1cb4e672b0eccf2bcdfbf835e32ff2a6d233a70eb33f29db" Sep 10 17:47:28.383722 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:47:28.383693 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98f91a3795cc47ac1cb4e672b0eccf2bcdfbf835e32ff2a6d233a70eb33f29db\": container with ID starting with 98f91a3795cc47ac1cb4e672b0eccf2bcdfbf835e32ff2a6d233a70eb33f29db not found: ID does not exist" containerID="98f91a3795cc47ac1cb4e672b0eccf2bcdfbf835e32ff2a6d233a70eb33f29db" Sep 10 17:47:28.383784 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.383737 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98f91a3795cc47ac1cb4e672b0eccf2bcdfbf835e32ff2a6d233a70eb33f29db"} err="failed to get container status \"98f91a3795cc47ac1cb4e672b0eccf2bcdfbf835e32ff2a6d233a70eb33f29db\": rpc error: code = NotFound desc = could not find container \"98f91a3795cc47ac1cb4e672b0eccf2bcdfbf835e32ff2a6d233a70eb33f29db\": container with ID starting with 98f91a3795cc47ac1cb4e672b0eccf2bcdfbf835e32ff2a6d233a70eb33f29db not found: ID does not exist" Sep 10 17:47:28.436340 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.436306 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p28bf\" (UniqueName: \"kubernetes.io/projected/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-kube-api-access-p28bf\") pod \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " Sep 10 17:47:28.436525 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.436354 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-csr-dns-cert\") pod \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " Sep 10 17:47:28.436525 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.436392 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-csr-ca-configmap\") pod \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " Sep 10 17:47:28.436525 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.436442 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-token\") pod \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " Sep 10 17:47:28.436525 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.436493 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-local-certs\") pod \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " Sep 10 17:47:28.436525 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.436526 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-kubeconfig\") pod \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " Sep 10 17:47:28.436770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.436570 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-cacerts\") pod \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\" (UID: \"dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce\") " Sep 10 17:47:28.436930 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.436870 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce" (UID: "dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:47:28.439181 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.439144 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-kube-api-access-p28bf" (OuterVolumeSpecName: "kube-api-access-p28bf") pod "dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce" (UID: "dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce"). InnerVolumeSpecName "kube-api-access-p28bf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:47:28.439288 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.439226 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce" (UID: "dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:47:28.439288 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.439242 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-token" (OuterVolumeSpecName: "istio-token") pod "dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce" (UID: "dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:47:28.439288 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.439238 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-local-certs" (OuterVolumeSpecName: "local-certs") pod "dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce" (UID: "dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:47:28.439399 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.439327 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce" (UID: "dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:47:28.439399 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.439388 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-cacerts" (OuterVolumeSpecName: "cacerts") pod "dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce" (UID: "dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:47:28.538020 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.537985 2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-csr-dns-cert\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:47:28.538020 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.538014 2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-csr-ca-configmap\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:47:28.538020 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.538025 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-token\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:47:28.538455 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.538034 2565 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-local-certs\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:47:28.538455 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.538041 2565 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-istio-kubeconfig\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:47:28.538455 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.538048 2565 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-cacerts\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:47:28.538455 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.538056 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p28bf\" (UniqueName: \"kubernetes.io/projected/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce-kube-api-access-p28bf\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:47:28.703115 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.703077 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g"] Sep 10 17:47:28.710956 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.710906 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r4v2g"] Sep 10 17:47:28.773342 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.773313 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:28.882832 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:28.882796 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:29.040978 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:29.040900 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:30.438645 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:30.438608 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce" path="/var/lib/kubelet/pods/dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce/volumes" Sep 10 17:47:31.393844 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:31.393804 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8" event={"ID":"f179b291-78a4-416a-9950-d04c61a061fc","Type":"ContainerStarted","Data":"149eef68abc34123b516b679a6bf72421abde1f4c3030f39c2a71d6bf15a52e0"} Sep 10 17:47:31.394057 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:31.393881 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8" Sep 10 17:47:31.395592 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:31.395566 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8" event={"ID":"7915f0f9-6e25-48a0-b177-0b406b6c6d87","Type":"ContainerStarted","Data":"eefc0698b8cd778810bb28b48c6f1e6f7fe056d7e8286080a9ac0a17ef7a80f4"} Sep 10 17:47:31.395694 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:31.395622 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8" Sep 10 17:47:31.411209 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:31.411151 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8" podStartSLOduration=1.415088114 podStartE2EDuration="4.41113407s" totalImagesPullingTime="2.996045956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:47:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:47:31.409366766 +0000 UTC m=+455.473777499" watchObservedRunningTime="2026-09-10 17:47:31.41113407 +0000 UTC m=+455.475544781" Sep 10 17:47:31.425821 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:31.425762 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8" podStartSLOduration=1.43487231 podStartE2EDuration="4.425744757s" totalImagesPullingTime="2.990872447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:47:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:47:31.424345059 +0000 UTC m=+455.488755771" watchObservedRunningTime="2026-09-10 17:47:31.425744757 +0000 UTC m=+455.490155467" Sep 10 17:47:36.908030 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:36.907980 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68699b7576-q6c5r"] Sep 10 17:47:36.908449 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:36.908329 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce" containerName="discovery" Sep 10 17:47:36.908449 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:36.908342 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce" containerName="discovery" Sep 10 17:47:36.908449 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:36.908413 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfd9dada-0f13-4a89-9a3e-1ae69f85d0ce" containerName="discovery" Sep 10 17:47:36.911362 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:36.911341 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:36.915331 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:36.915308 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 17:47:36.915457 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:36.915369 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 17:47:36.919486 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:36.919467 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 17:47:36.919915 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:36.919876 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 17:47:36.919997 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:36.919983 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9s9rn\"" Sep 10 17:47:36.920070 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:36.919988 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 17:47:36.923187 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:36.923161 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 17:47:36.925039 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:36.925022 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68699b7576-q6c5r"] Sep 10 17:47:36.963985 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:36.963953 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:37.008953 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.008913 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c879dd63-c950-44b3-ab60-7921c4b728d2-console-oauth-config\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.009138 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.008987 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c879dd63-c950-44b3-ab60-7921c4b728d2-console-config\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.009138 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.009049 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c879dd63-c950-44b3-ab60-7921c4b728d2-trusted-ca-bundle\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.009138 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.009099 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c879dd63-c950-44b3-ab60-7921c4b728d2-service-ca\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.009341 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.009139 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c879dd63-c950-44b3-ab60-7921c4b728d2-console-serving-cert\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.009341 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.009163 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q4vb\" (UniqueName: \"kubernetes.io/projected/c879dd63-c950-44b3-ab60-7921c4b728d2-kube-api-access-6q4vb\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.009341 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.009249 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c879dd63-c950-44b3-ab60-7921c4b728d2-oauth-serving-cert\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.109817 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.109783 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c879dd63-c950-44b3-ab60-7921c4b728d2-service-ca\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.109817 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.109830 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c879dd63-c950-44b3-ab60-7921c4b728d2-console-serving-cert\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.110058 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.109853 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q4vb\" (UniqueName: \"kubernetes.io/projected/c879dd63-c950-44b3-ab60-7921c4b728d2-kube-api-access-6q4vb\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.110058 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.109880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c879dd63-c950-44b3-ab60-7921c4b728d2-oauth-serving-cert\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.110058 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.109931 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c879dd63-c950-44b3-ab60-7921c4b728d2-console-oauth-config\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.110058 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.109971 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c879dd63-c950-44b3-ab60-7921c4b728d2-console-config\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.110058 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.110011 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c879dd63-c950-44b3-ab60-7921c4b728d2-trusted-ca-bundle\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.110615 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.110585 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c879dd63-c950-44b3-ab60-7921c4b728d2-service-ca\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.110753 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.110676 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c879dd63-c950-44b3-ab60-7921c4b728d2-oauth-serving-cert\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.110753 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.110721 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c879dd63-c950-44b3-ab60-7921c4b728d2-console-config\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.110881 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.110862 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c879dd63-c950-44b3-ab60-7921c4b728d2-trusted-ca-bundle\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.112491 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.112467 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c879dd63-c950-44b3-ab60-7921c4b728d2-console-oauth-config\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.112645 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.112584 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c879dd63-c950-44b3-ab60-7921c4b728d2-console-serving-cert\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.117715 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.117689 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q4vb\" (UniqueName: \"kubernetes.io/projected/c879dd63-c950-44b3-ab60-7921c4b728d2-kube-api-access-6q4vb\") pod \"console-68699b7576-q6c5r\" (UID: \"c879dd63-c950-44b3-ab60-7921c4b728d2\") " pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.221775 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.221678 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:37.497251 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:37.497222 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68699b7576-q6c5r"] Sep 10 17:47:37.499382 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:47:37.499349 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc879dd63_c950_44b3_ab60_7921c4b728d2.slice/crio-45e1cb4c7e57e4569b9c16003c0fe4a065a3f2f70050cb59d82d04c4c9aaf559 WatchSource:0}: Error finding container 45e1cb4c7e57e4569b9c16003c0fe4a065a3f2f70050cb59d82d04c4c9aaf559: Status 404 returned error can't find the container with id 45e1cb4c7e57e4569b9c16003c0fe4a065a3f2f70050cb59d82d04c4c9aaf559 Sep 10 17:47:38.427946 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:38.427913 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68699b7576-q6c5r" event={"ID":"c879dd63-c950-44b3-ab60-7921c4b728d2","Type":"ContainerStarted","Data":"8123579841dbc62a19316e440a6ea858761f812f18fa24298ab53c3072a45288"} Sep 10 17:47:38.428335 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:38.427960 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68699b7576-q6c5r" event={"ID":"c879dd63-c950-44b3-ab60-7921c4b728d2","Type":"ContainerStarted","Data":"45e1cb4c7e57e4569b9c16003c0fe4a065a3f2f70050cb59d82d04c4c9aaf559"} Sep 10 17:47:38.445968 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:38.445916 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68699b7576-q6c5r" podStartSLOduration=2.445900425 podStartE2EDuration="2.445900425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:47:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:47:38.444364879 +0000 UTC m=+462.508775607" watchObservedRunningTime="2026-09-10 17:47:38.445900425 +0000 UTC m=+462.510311156" Sep 10 17:47:47.222838 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:47.222799 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:47.223243 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:47.222861 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:47.227614 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:47.227590 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:47.470331 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:47.470297 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68699b7576-q6c5r" Sep 10 17:47:47.623217 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:47.623138 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:47.724513 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:47.724474 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:47:47.835562 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:47:47.835532 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:02.404060 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:02.404027 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jj5r8" Sep 10 17:48:02.406881 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:02.406858 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4wwx8" Sep 10 17:48:18.857248 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:18.857204 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:19.039897 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:19.039863 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:45.821717 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:45.821679 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:47.848766 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:47.848732 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fczpx"] Sep 10 17:48:47.851487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:47.851468 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fczpx" Sep 10 17:48:47.853599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:47.853578 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 10 17:48:47.853599 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:47.853591 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 10 17:48:47.853757 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:47.853578 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-d5d7z\"" Sep 10 17:48:47.853757 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:47.853578 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 10 17:48:47.861758 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:47.861735 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fczpx"] Sep 10 17:48:47.981383 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:47.981351 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj2ls\" (UniqueName: \"kubernetes.io/projected/e6e60860-6b9f-4969-973a-5e68a1da9400-kube-api-access-pj2ls\") pod \"dns-operator-controller-manager-7b84cd7c9c-fczpx\" (UID: \"e6e60860-6b9f-4969-973a-5e68a1da9400\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fczpx" Sep 10 17:48:48.082471 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:48.082441 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj2ls\" (UniqueName: \"kubernetes.io/projected/e6e60860-6b9f-4969-973a-5e68a1da9400-kube-api-access-pj2ls\") pod \"dns-operator-controller-manager-7b84cd7c9c-fczpx\" (UID: \"e6e60860-6b9f-4969-973a-5e68a1da9400\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fczpx" Sep 10 17:48:48.095029 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:48.095001 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj2ls\" (UniqueName: \"kubernetes.io/projected/e6e60860-6b9f-4969-973a-5e68a1da9400-kube-api-access-pj2ls\") pod \"dns-operator-controller-manager-7b84cd7c9c-fczpx\" (UID: \"e6e60860-6b9f-4969-973a-5e68a1da9400\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fczpx" Sep 10 17:48:48.161497 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:48.161472 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fczpx" Sep 10 17:48:48.209278 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:48.209242 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:48.435961 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:48:48.435927 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6e60860_6b9f_4969_973a_5e68a1da9400.slice/crio-4576c6505d144a4469c130dce0ee16d5437d391a02200a09052eb80a0a45930b WatchSource:0}: Error finding container 4576c6505d144a4469c130dce0ee16d5437d391a02200a09052eb80a0a45930b: Status 404 returned error can't find the container with id 4576c6505d144a4469c130dce0ee16d5437d391a02200a09052eb80a0a45930b Sep 10 17:48:48.438402 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:48.438378 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fczpx"] Sep 10 17:48:48.682138 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:48.682094 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:48.720128 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:48.720053 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fczpx" event={"ID":"e6e60860-6b9f-4969-973a-5e68a1da9400","Type":"ContainerStarted","Data":"4576c6505d144a4469c130dce0ee16d5437d391a02200a09052eb80a0a45930b"} Sep 10 17:48:50.073784 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:50.073748 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:50.438629 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:50.438602 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-jb2np"] Sep 10 17:48:50.443371 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:50.443344 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-jb2np" Sep 10 17:48:50.447497 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:50.447469 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-rmflq\"" Sep 10 17:48:50.467138 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:50.467108 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-jb2np"] Sep 10 17:48:50.506049 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:50.506016 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6ntr\" (UniqueName: \"kubernetes.io/projected/0b932e8b-986e-4d45-870c-9b9b1d12d1f1-kube-api-access-j6ntr\") pod \"authorino-operator-76b478bd78-jb2np\" (UID: \"0b932e8b-986e-4d45-870c-9b9b1d12d1f1\") " pod="kuadrant-system/authorino-operator-76b478bd78-jb2np" Sep 10 17:48:50.606495 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:50.606466 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6ntr\" (UniqueName: \"kubernetes.io/projected/0b932e8b-986e-4d45-870c-9b9b1d12d1f1-kube-api-access-j6ntr\") pod \"authorino-operator-76b478bd78-jb2np\" (UID: \"0b932e8b-986e-4d45-870c-9b9b1d12d1f1\") " pod="kuadrant-system/authorino-operator-76b478bd78-jb2np" Sep 10 17:48:50.618764 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:50.618731 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6ntr\" (UniqueName: \"kubernetes.io/projected/0b932e8b-986e-4d45-870c-9b9b1d12d1f1-kube-api-access-j6ntr\") pod \"authorino-operator-76b478bd78-jb2np\" (UID: \"0b932e8b-986e-4d45-870c-9b9b1d12d1f1\") " pod="kuadrant-system/authorino-operator-76b478bd78-jb2np" Sep 10 17:48:50.757186 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:50.757115 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-jb2np" Sep 10 17:48:51.049849 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:51.049822 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:51.260440 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:51.260417 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-jb2np"] Sep 10 17:48:51.264693 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:48:51.264668 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b932e8b_986e_4d45_870c_9b9b1d12d1f1.slice/crio-3371c686e8905406d4814da402c622fc24d84a51c4309eb3d84c7c229beef786 WatchSource:0}: Error finding container 3371c686e8905406d4814da402c622fc24d84a51c4309eb3d84c7c229beef786: Status 404 returned error can't find the container with id 3371c686e8905406d4814da402c622fc24d84a51c4309eb3d84c7c229beef786 Sep 10 17:48:51.312140 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:51.312085 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:51.417545 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:51.417502 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:51.738487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:51.738450 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fczpx" event={"ID":"e6e60860-6b9f-4969-973a-5e68a1da9400","Type":"ContainerStarted","Data":"b2f754777d37d2fb6469d99402722e52c2792fd2fd21fbb28279a09f67e885d2"} Sep 10 17:48:51.738674 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:51.738585 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fczpx" Sep 10 17:48:51.740056 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:51.740036 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-jb2np" event={"ID":"0b932e8b-986e-4d45-870c-9b9b1d12d1f1","Type":"ContainerStarted","Data":"3371c686e8905406d4814da402c622fc24d84a51c4309eb3d84c7c229beef786"} Sep 10 17:48:51.760581 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:51.760529 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fczpx" podStartSLOduration=2.145215724 podStartE2EDuration="4.760515975s" totalImagesPullingTime="2.615300251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:48:51.757458781 +0000 UTC m=+535.821869492" watchObservedRunningTime="2026-09-10 17:48:51.760515975 +0000 UTC m=+535.824926686" Sep 10 17:48:52.002006 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:52.001936 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:52.746430 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:52.746397 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-jb2np" event={"ID":"0b932e8b-986e-4d45-870c-9b9b1d12d1f1","Type":"ContainerStarted","Data":"5679814434c0980ea562370ef594b4d49de235fe7d9f64162bc8747b74af23c4"} Sep 10 17:48:52.771538 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:52.771454 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-jb2np" podStartSLOduration=1.520425422 podStartE2EDuration="2.771438052s" totalImagesPullingTime="1.25101263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:48:52.76946036 +0000 UTC m=+536.833871089" watchObservedRunningTime="2026-09-10 17:48:52.771438052 +0000 UTC m=+536.835848762" Sep 10 17:48:52.907923 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:52.907878 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:53.017564 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:53.017533 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:53.750520 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:53.750488 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-jb2np" Sep 10 17:48:54.959108 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:54.959075 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2"] Sep 10 17:48:54.962702 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:54.962683 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:48:54.964810 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:54.964786 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gp9gs\"" Sep 10 17:48:54.977615 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:54.977595 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2"] Sep 10 17:48:55.042808 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:55.042782 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58aa3a28-d38a-4b32-8610-33aeabd91934-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" (UID: \"58aa3a28-d38a-4b32-8610-33aeabd91934\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:48:55.042953 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:55.042830 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlb6v\" (UniqueName: \"kubernetes.io/projected/58aa3a28-d38a-4b32-8610-33aeabd91934-kube-api-access-xlb6v\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" (UID: \"58aa3a28-d38a-4b32-8610-33aeabd91934\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:48:55.143360 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:55.143331 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58aa3a28-d38a-4b32-8610-33aeabd91934-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" (UID: \"58aa3a28-d38a-4b32-8610-33aeabd91934\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:48:55.143554 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:55.143390 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlb6v\" (UniqueName: \"kubernetes.io/projected/58aa3a28-d38a-4b32-8610-33aeabd91934-kube-api-access-xlb6v\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" (UID: \"58aa3a28-d38a-4b32-8610-33aeabd91934\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:48:55.143691 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:55.143658 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58aa3a28-d38a-4b32-8610-33aeabd91934-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" (UID: \"58aa3a28-d38a-4b32-8610-33aeabd91934\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:48:55.152714 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:55.152689 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlb6v\" (UniqueName: \"kubernetes.io/projected/58aa3a28-d38a-4b32-8610-33aeabd91934-kube-api-access-xlb6v\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" (UID: \"58aa3a28-d38a-4b32-8610-33aeabd91934\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:48:55.271967 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:55.271909 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:48:55.464175 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:55.464122 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:55.557079 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:55.557057 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2"] Sep 10 17:48:55.558603 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:48:55.558583 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58aa3a28_d38a_4b32_8610_33aeabd91934.slice/crio-5f7e1a217ee257a4263d3f52c2d94b5f1fb05e6e99065e12c20fd8fbe7daad32 WatchSource:0}: Error finding container 5f7e1a217ee257a4263d3f52c2d94b5f1fb05e6e99065e12c20fd8fbe7daad32: Status 404 returned error can't find the container with id 5f7e1a217ee257a4263d3f52c2d94b5f1fb05e6e99065e12c20fd8fbe7daad32 Sep 10 17:48:55.613373 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:55.613348 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:55.760467 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:55.760426 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" event={"ID":"58aa3a28-d38a-4b32-8610-33aeabd91934","Type":"ContainerStarted","Data":"5f7e1a217ee257a4263d3f52c2d94b5f1fb05e6e99065e12c20fd8fbe7daad32"} Sep 10 17:48:57.225297 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:57.225259 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm"] Sep 10 17:48:57.229075 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:57.229053 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:48:57.232936 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:57.232916 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-cq44z\"" Sep 10 17:48:57.244050 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:57.243996 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm"] Sep 10 17:48:57.366897 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:57.366862 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drtv7\" (UniqueName: \"kubernetes.io/projected/ec9e0601-acc4-4456-bfc9-bde2d819f0b6-kube-api-access-drtv7\") pod \"limitador-operator-controller-manager-74495c69f8-mqsdm\" (UID: \"ec9e0601-acc4-4456-bfc9-bde2d819f0b6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:48:57.467904 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:57.467849 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drtv7\" (UniqueName: \"kubernetes.io/projected/ec9e0601-acc4-4456-bfc9-bde2d819f0b6-kube-api-access-drtv7\") pod \"limitador-operator-controller-manager-74495c69f8-mqsdm\" (UID: \"ec9e0601-acc4-4456-bfc9-bde2d819f0b6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:48:57.482506 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:57.482446 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drtv7\" (UniqueName: \"kubernetes.io/projected/ec9e0601-acc4-4456-bfc9-bde2d819f0b6-kube-api-access-drtv7\") pod \"limitador-operator-controller-manager-74495c69f8-mqsdm\" (UID: \"ec9e0601-acc4-4456-bfc9-bde2d819f0b6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:48:57.541536 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:57.541487 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:48:57.820420 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:57.820394 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm"] Sep 10 17:48:58.427447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:58.427403 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:58.585491 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:58.585462 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:48:58.885780 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:48:58.885749 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec9e0601_acc4_4456_bfc9_bde2d819f0b6.slice/crio-22035667e64764ff85681101b904ec1fe381b926f629708e2b913acc12c4fd48 WatchSource:0}: Error finding container 22035667e64764ff85681101b904ec1fe381b926f629708e2b913acc12c4fd48: Status 404 returned error can't find the container with id 22035667e64764ff85681101b904ec1fe381b926f629708e2b913acc12c4fd48 Sep 10 17:48:59.782805 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:59.782770 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" event={"ID":"ec9e0601-acc4-4456-bfc9-bde2d819f0b6","Type":"ContainerStarted","Data":"22035667e64764ff85681101b904ec1fe381b926f629708e2b913acc12c4fd48"} Sep 10 17:48:59.784715 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:59.784687 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" event={"ID":"58aa3a28-d38a-4b32-8610-33aeabd91934","Type":"ContainerStarted","Data":"b9c02ade3aa4705cca71e9edb4111c41486f4623b3bd1340a381de69fe73cbfb"} Sep 10 17:48:59.784858 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:59.784842 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:48:59.807881 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:48:59.807824 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" podStartSLOduration=2.414008499 podStartE2EDuration="5.807806176s" totalImagesPullingTime="3.393797677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:48:59.80540295 +0000 UTC m=+543.869813682" watchObservedRunningTime="2026-09-10 17:48:59.807806176 +0000 UTC m=+543.872216889" Sep 10 17:49:00.791390 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:00.791350 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" event={"ID":"ec9e0601-acc4-4456-bfc9-bde2d819f0b6","Type":"ContainerStarted","Data":"d0ea60bb7a2604c09e3d1f8e7906fcb02c8513a462af5375217fd2a16797afad"} Sep 10 17:49:00.791831 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:00.791425 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:49:00.809138 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:00.809097 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" podStartSLOduration=2.665796142 podStartE2EDuration="3.809083972s" totalImagesPullingTime="1.14328783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:00.807657506 +0000 UTC m=+544.872068238" watchObservedRunningTime="2026-09-10 17:49:00.809083972 +0000 UTC m=+544.873494682" Sep 10 17:49:01.609441 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.609412 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv"] Sep 10 17:49:01.612922 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.612902 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" Sep 10 17:49:01.614827 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.614795 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Sep 10 17:49:01.615520 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.615503 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Sep 10 17:49:01.615653 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.615636 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fmjnf\"" Sep 10 17:49:01.620308 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.620160 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv"] Sep 10 17:49:01.625239 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.625217 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:01.700845 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.700821 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/803b68d9-477d-40ee-b4a9-12e748a24384-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-5x7xv\" (UID: \"803b68d9-477d-40ee-b4a9-12e748a24384\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" Sep 10 17:49:01.700972 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.700868 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/803b68d9-477d-40ee-b4a9-12e748a24384-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-5x7xv\" (UID: \"803b68d9-477d-40ee-b4a9-12e748a24384\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" Sep 10 17:49:01.700972 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.700950 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22v2b\" (UniqueName: \"kubernetes.io/projected/803b68d9-477d-40ee-b4a9-12e748a24384-kube-api-access-22v2b\") pod \"kuadrant-console-plugin-fbdd4f445-5x7xv\" (UID: \"803b68d9-477d-40ee-b4a9-12e748a24384\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" Sep 10 17:49:01.801294 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.801271 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/803b68d9-477d-40ee-b4a9-12e748a24384-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-5x7xv\" (UID: \"803b68d9-477d-40ee-b4a9-12e748a24384\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" Sep 10 17:49:01.801596 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.801312 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/803b68d9-477d-40ee-b4a9-12e748a24384-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-5x7xv\" (UID: \"803b68d9-477d-40ee-b4a9-12e748a24384\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" Sep 10 17:49:01.801596 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.801340 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22v2b\" (UniqueName: \"kubernetes.io/projected/803b68d9-477d-40ee-b4a9-12e748a24384-kube-api-access-22v2b\") pod \"kuadrant-console-plugin-fbdd4f445-5x7xv\" (UID: \"803b68d9-477d-40ee-b4a9-12e748a24384\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" Sep 10 17:49:01.801596 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:01.801413 2565 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Sep 10 17:49:01.801596 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:01.801466 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/803b68d9-477d-40ee-b4a9-12e748a24384-plugin-serving-cert podName:803b68d9-477d-40ee-b4a9-12e748a24384 nodeName:}" failed. No retries permitted until 2026-09-10 17:49:02.301452076 +0000 UTC m=+546.365862779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/803b68d9-477d-40ee-b4a9-12e748a24384-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-5x7xv" (UID: "803b68d9-477d-40ee-b4a9-12e748a24384") : secret "plugin-serving-cert" not found Sep 10 17:49:01.802024 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.802006 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/803b68d9-477d-40ee-b4a9-12e748a24384-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-5x7xv\" (UID: \"803b68d9-477d-40ee-b4a9-12e748a24384\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" Sep 10 17:49:01.809984 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.809966 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22v2b\" (UniqueName: \"kubernetes.io/projected/803b68d9-477d-40ee-b4a9-12e748a24384-kube-api-access-22v2b\") pod \"kuadrant-console-plugin-fbdd4f445-5x7xv\" (UID: \"803b68d9-477d-40ee-b4a9-12e748a24384\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" Sep 10 17:49:01.910747 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:01.910721 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:02.306120 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:02.306038 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/803b68d9-477d-40ee-b4a9-12e748a24384-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-5x7xv\" (UID: \"803b68d9-477d-40ee-b4a9-12e748a24384\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" Sep 10 17:49:02.308394 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:02.308376 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/803b68d9-477d-40ee-b4a9-12e748a24384-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-5x7xv\" (UID: \"803b68d9-477d-40ee-b4a9-12e748a24384\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" Sep 10 17:49:02.522680 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:02.522645 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" Sep 10 17:49:02.748627 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:02.748600 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-fczpx" Sep 10 17:49:02.794655 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:02.794523 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv"] Sep 10 17:49:02.796828 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:49:02.796798 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod803b68d9_477d_40ee_b4a9_12e748a24384.slice/crio-cf1baf30610a10ece114ae86194f1d53d0d73767b75ec7121961b4c4fe64cb7b WatchSource:0}: Error finding container cf1baf30610a10ece114ae86194f1d53d0d73767b75ec7121961b4c4fe64cb7b: Status 404 returned error can't find the container with id cf1baf30610a10ece114ae86194f1d53d0d73767b75ec7121961b4c4fe64cb7b Sep 10 17:49:03.623178 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:03.623117 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:03.725589 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:03.725559 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:03.809382 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:03.809349 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" event={"ID":"803b68d9-477d-40ee-b4a9-12e748a24384","Type":"ContainerStarted","Data":"cf1baf30610a10ece114ae86194f1d53d0d73767b75ec7121961b4c4fe64cb7b"} Sep 10 17:49:04.014234 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:04.014203 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:04.111005 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:04.110975 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:04.518022 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:04.517992 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:04.632499 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:04.632470 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:04.757861 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:04.757828 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-jb2np" Sep 10 17:49:07.831975 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:07.831942 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" event={"ID":"803b68d9-477d-40ee-b4a9-12e748a24384","Type":"ContainerStarted","Data":"b082cf402abf124649c7e90daaa869bf1392ba55e40887128b2e0e53270d31ed"} Sep 10 17:49:07.861086 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:07.861047 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5x7xv" podStartSLOduration=2.686664991 podStartE2EDuration="6.861030411s" totalImagesPullingTime="4.17436542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:07.860068537 +0000 UTC m=+551.924479249" watchObservedRunningTime="2026-09-10 17:49:07.861030411 +0000 UTC m=+551.925441122" Sep 10 17:49:08.288253 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:08.288223 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:08.388435 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:08.388406 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:08.487942 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:08.487913 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:10.793819 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:10.793791 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:49:11.797624 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:11.797593 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:49:12.023446 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.023409 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq"] Sep 10 17:49:12.027134 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.027119 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:12.039512 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.039487 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq"] Sep 10 17:49:12.189675 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.189639 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-4cnxq\" (UID: \"2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:12.189675 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.189677 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vnrs\" (UniqueName: \"kubernetes.io/projected/2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43-kube-api-access-9vnrs\") pod \"kuadrant-operator-controller-manager-5f685c496f-4cnxq\" (UID: \"2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:12.290484 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.290447 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-4cnxq\" (UID: \"2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:12.290484 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.290483 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vnrs\" (UniqueName: \"kubernetes.io/projected/2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43-kube-api-access-9vnrs\") pod \"kuadrant-operator-controller-manager-5f685c496f-4cnxq\" (UID: \"2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:12.290837 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.290818 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-4cnxq\" (UID: \"2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:12.301463 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.301432 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vnrs\" (UniqueName: \"kubernetes.io/projected/2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43-kube-api-access-9vnrs\") pod \"kuadrant-operator-controller-manager-5f685c496f-4cnxq\" (UID: \"2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:12.337221 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.337194 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:12.634632 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.634606 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq"] Sep 10 17:49:12.636794 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:49:12.636771 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a2ee932_49d2_4cb3_a263_9bf3cc8dbb43.slice/crio-1ebc95483238e2bcb25d111c1a51fffebc2e262c75ed996df558d04272f0deb5 WatchSource:0}: Error finding container 1ebc95483238e2bcb25d111c1a51fffebc2e262c75ed996df558d04272f0deb5: Status 404 returned error can't find the container with id 1ebc95483238e2bcb25d111c1a51fffebc2e262c75ed996df558d04272f0deb5 Sep 10 17:49:12.857485 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.857451 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" event={"ID":"2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43","Type":"ContainerStarted","Data":"c10e868fd58ac64f3729f65e1db742710778ecae50934f2718aff06c2d7a1d8f"} Sep 10 17:49:12.857833 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.857508 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" event={"ID":"2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43","Type":"ContainerStarted","Data":"1ebc95483238e2bcb25d111c1a51fffebc2e262c75ed996df558d04272f0deb5"} Sep 10 17:49:12.857833 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.857622 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:12.881437 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:12.881390 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" podStartSLOduration=0.881377034 podStartE2EDuration="881.377034ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:12.879983903 +0000 UTC m=+556.944394661" watchObservedRunningTime="2026-09-10 17:49:12.881377034 +0000 UTC m=+556.945787744" Sep 10 17:49:13.045675 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.045589 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq"] Sep 10 17:49:13.055789 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.055758 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq"] Sep 10 17:49:13.071783 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.071752 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct"] Sep 10 17:49:13.072136 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.072122 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43" containerName="manager" Sep 10 17:49:13.072201 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.072137 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43" containerName="manager" Sep 10 17:49:13.072201 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.072199 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43" containerName="manager" Sep 10 17:49:13.075095 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.075076 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2"] Sep 10 17:49:13.075231 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.075209 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" podUID="58aa3a28-d38a-4b32-8610-33aeabd91934" containerName="manager" containerID="cri-o://b9c02ade3aa4705cca71e9edb4111c41486f4623b3bd1340a381de69fe73cbfb" gracePeriod=2 Sep 10 17:49:13.075296 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.075238 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" Sep 10 17:49:13.089245 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.089224 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2"] Sep 10 17:49:13.094538 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.094460 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct"] Sep 10 17:49:13.096781 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.096612 2565 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"58aa3a28-d38a-4b32-8610-33aeabd91934\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-wckq2\": pods \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:49:13.098348 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.098321 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="58aa3a28-d38a-4b32-8610-33aeabd91934" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:49:13.105509 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.105488 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf"] Sep 10 17:49:13.105868 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.105852 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58aa3a28-d38a-4b32-8610-33aeabd91934" containerName="manager" Sep 10 17:49:13.105956 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.105872 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="58aa3a28-d38a-4b32-8610-33aeabd91934" containerName="manager" Sep 10 17:49:13.106019 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.105998 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="58aa3a28-d38a-4b32-8610-33aeabd91934" containerName="manager" Sep 10 17:49:13.109041 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.109022 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm"] Sep 10 17:49:13.109147 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.109130 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" Sep 10 17:49:13.109191 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.109150 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" podUID="ec9e0601-acc4-4456-bfc9-bde2d819f0b6" containerName="manager" containerID="cri-o://d0ea60bb7a2604c09e3d1f8e7906fcb02c8513a462af5375217fd2a16797afad" gracePeriod=2 Sep 10 17:49:13.122855 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.122832 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm"] Sep 10 17:49:13.127343 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.127016 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="58aa3a28-d38a-4b32-8610-33aeabd91934" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:49:13.128795 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.128767 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf"] Sep 10 17:49:13.134910 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.134872 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-qbl8m"] Sep 10 17:49:13.135233 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.135221 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec9e0601-acc4-4456-bfc9-bde2d819f0b6" containerName="manager" Sep 10 17:49:13.135233 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.135234 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec9e0601-acc4-4456-bfc9-bde2d819f0b6" containerName="manager" Sep 10 17:49:13.135342 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.135293 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec9e0601-acc4-4456-bfc9-bde2d819f0b6" containerName="manager" Sep 10 17:49:13.138073 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.138056 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qbl8m" Sep 10 17:49:13.152904 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.152856 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-qbl8m"] Sep 10 17:49:13.153516 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.153329 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mqsdm\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="ec9e0601-acc4-4456-bfc9-bde2d819f0b6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:49:13.154999 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.154869 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mqsdm\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="ec9e0601-acc4-4456-bfc9-bde2d819f0b6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:49:13.156368 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.156344 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="58aa3a28-d38a-4b32-8610-33aeabd91934" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:49:13.197924 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.197864 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv287\" (UniqueName: \"kubernetes.io/projected/21fa1895-2079-45ea-bf65-78a26648b98b-kube-api-access-xv287\") pod \"kuadrant-operator-controller-manager-5f685c496f-wrpct\" (UID: \"21fa1895-2079-45ea-bf65-78a26648b98b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" Sep 10 17:49:13.198039 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.197974 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21fa1895-2079-45ea-bf65-78a26648b98b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-wrpct\" (UID: \"21fa1895-2079-45ea-bf65-78a26648b98b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" Sep 10 17:49:13.299440 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.299378 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv287\" (UniqueName: \"kubernetes.io/projected/21fa1895-2079-45ea-bf65-78a26648b98b-kube-api-access-xv287\") pod \"kuadrant-operator-controller-manager-5f685c496f-wrpct\" (UID: \"21fa1895-2079-45ea-bf65-78a26648b98b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" Sep 10 17:49:13.299554 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.299442 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21fa1895-2079-45ea-bf65-78a26648b98b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-wrpct\" (UID: \"21fa1895-2079-45ea-bf65-78a26648b98b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" Sep 10 17:49:13.299554 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.299491 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgnjt\" (UniqueName: \"kubernetes.io/projected/0ebf63a2-b8ce-4cb5-94ae-659e860c1635-kube-api-access-vgnjt\") pod \"kuadrant-operator-controller-manager-76f8f686bf-76jjf\" (UID: \"0ebf63a2-b8ce-4cb5-94ae-659e860c1635\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" Sep 10 17:49:13.299554 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.299534 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr4xf\" (UniqueName: \"kubernetes.io/projected/b8be19ca-3248-451c-bc93-b4eebb5e38bd-kube-api-access-hr4xf\") pod \"limitador-operator-controller-manager-74495c69f8-qbl8m\" (UID: \"b8be19ca-3248-451c-bc93-b4eebb5e38bd\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qbl8m" Sep 10 17:49:13.299691 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.299556 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ebf63a2-b8ce-4cb5-94ae-659e860c1635-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-76jjf\" (UID: \"0ebf63a2-b8ce-4cb5-94ae-659e860c1635\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" Sep 10 17:49:13.299931 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.299907 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21fa1895-2079-45ea-bf65-78a26648b98b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-wrpct\" (UID: \"21fa1895-2079-45ea-bf65-78a26648b98b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" Sep 10 17:49:13.311949 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.311923 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv287\" (UniqueName: \"kubernetes.io/projected/21fa1895-2079-45ea-bf65-78a26648b98b-kube-api-access-xv287\") pod \"kuadrant-operator-controller-manager-5f685c496f-wrpct\" (UID: \"21fa1895-2079-45ea-bf65-78a26648b98b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" Sep 10 17:49:13.350788 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.350766 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:49:13.352756 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.352729 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mqsdm\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="ec9e0601-acc4-4456-bfc9-bde2d819f0b6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:49:13.353447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.353431 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:49:13.356143 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.356115 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="58aa3a28-d38a-4b32-8610-33aeabd91934" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:49:13.358063 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.358039 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mqsdm\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="ec9e0601-acc4-4456-bfc9-bde2d819f0b6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:49:13.359528 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.359504 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="58aa3a28-d38a-4b32-8610-33aeabd91934" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:49:13.386091 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.386067 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" Sep 10 17:49:13.401274 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.401249 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgnjt\" (UniqueName: \"kubernetes.io/projected/0ebf63a2-b8ce-4cb5-94ae-659e860c1635-kube-api-access-vgnjt\") pod \"kuadrant-operator-controller-manager-76f8f686bf-76jjf\" (UID: \"0ebf63a2-b8ce-4cb5-94ae-659e860c1635\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" Sep 10 17:49:13.401357 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.401310 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr4xf\" (UniqueName: \"kubernetes.io/projected/b8be19ca-3248-451c-bc93-b4eebb5e38bd-kube-api-access-hr4xf\") pod \"limitador-operator-controller-manager-74495c69f8-qbl8m\" (UID: \"b8be19ca-3248-451c-bc93-b4eebb5e38bd\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qbl8m" Sep 10 17:49:13.401357 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.401337 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ebf63a2-b8ce-4cb5-94ae-659e860c1635-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-76jjf\" (UID: \"0ebf63a2-b8ce-4cb5-94ae-659e860c1635\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" Sep 10 17:49:13.401729 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.401708 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ebf63a2-b8ce-4cb5-94ae-659e860c1635-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-76jjf\" (UID: \"0ebf63a2-b8ce-4cb5-94ae-659e860c1635\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" Sep 10 17:49:13.418526 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.418503 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgnjt\" (UniqueName: \"kubernetes.io/projected/0ebf63a2-b8ce-4cb5-94ae-659e860c1635-kube-api-access-vgnjt\") pod \"kuadrant-operator-controller-manager-76f8f686bf-76jjf\" (UID: \"0ebf63a2-b8ce-4cb5-94ae-659e860c1635\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" Sep 10 17:49:13.418622 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.418543 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr4xf\" (UniqueName: \"kubernetes.io/projected/b8be19ca-3248-451c-bc93-b4eebb5e38bd-kube-api-access-hr4xf\") pod \"limitador-operator-controller-manager-74495c69f8-qbl8m\" (UID: \"b8be19ca-3248-451c-bc93-b4eebb5e38bd\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qbl8m" Sep 10 17:49:13.497848 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.497801 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" Sep 10 17:49:13.501672 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.501649 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drtv7\" (UniqueName: \"kubernetes.io/projected/ec9e0601-acc4-4456-bfc9-bde2d819f0b6-kube-api-access-drtv7\") pod \"ec9e0601-acc4-4456-bfc9-bde2d819f0b6\" (UID: \"ec9e0601-acc4-4456-bfc9-bde2d819f0b6\") " Sep 10 17:49:13.501774 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.501733 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58aa3a28-d38a-4b32-8610-33aeabd91934-extensions-socket-volume\") pod \"58aa3a28-d38a-4b32-8610-33aeabd91934\" (UID: \"58aa3a28-d38a-4b32-8610-33aeabd91934\") " Sep 10 17:49:13.501830 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.501789 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlb6v\" (UniqueName: \"kubernetes.io/projected/58aa3a28-d38a-4b32-8610-33aeabd91934-kube-api-access-xlb6v\") pod \"58aa3a28-d38a-4b32-8610-33aeabd91934\" (UID: \"58aa3a28-d38a-4b32-8610-33aeabd91934\") " Sep 10 17:49:13.502320 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.502276 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58aa3a28-d38a-4b32-8610-33aeabd91934-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "58aa3a28-d38a-4b32-8610-33aeabd91934" (UID: "58aa3a28-d38a-4b32-8610-33aeabd91934"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:49:13.503861 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.503840 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qbl8m" Sep 10 17:49:13.503861 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.503847 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec9e0601-acc4-4456-bfc9-bde2d819f0b6-kube-api-access-drtv7" (OuterVolumeSpecName: "kube-api-access-drtv7") pod "ec9e0601-acc4-4456-bfc9-bde2d819f0b6" (UID: "ec9e0601-acc4-4456-bfc9-bde2d819f0b6"). InnerVolumeSpecName "kube-api-access-drtv7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:49:13.504033 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.503870 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58aa3a28-d38a-4b32-8610-33aeabd91934-kube-api-access-xlb6v" (OuterVolumeSpecName: "kube-api-access-xlb6v") pod "58aa3a28-d38a-4b32-8610-33aeabd91934" (UID: "58aa3a28-d38a-4b32-8610-33aeabd91934"). InnerVolumeSpecName "kube-api-access-xlb6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:49:13.603510 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.603481 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58aa3a28-d38a-4b32-8610-33aeabd91934-extensions-socket-volume\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:49:13.603510 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.603509 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlb6v\" (UniqueName: \"kubernetes.io/projected/58aa3a28-d38a-4b32-8610-33aeabd91934-kube-api-access-xlb6v\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:49:13.603670 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.603524 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drtv7\" (UniqueName: \"kubernetes.io/projected/ec9e0601-acc4-4456-bfc9-bde2d819f0b6-kube-api-access-drtv7\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:49:13.666666 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.666635 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct"] Sep 10 17:49:13.670778 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:49:13.670753 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21fa1895_2079_45ea_bf65_78a26648b98b.slice/crio-16e0ea7a682329459fd899318f34723379a67877395099ca355822defaf9a7b9 WatchSource:0}: Error finding container 16e0ea7a682329459fd899318f34723379a67877395099ca355822defaf9a7b9: Status 404 returned error can't find the container with id 16e0ea7a682329459fd899318f34723379a67877395099ca355822defaf9a7b9 Sep 10 17:49:13.776264 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.776211 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf"] Sep 10 17:49:13.778474 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.778450 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-qbl8m"] Sep 10 17:49:13.780078 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:49:13.780053 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ebf63a2_b8ce_4cb5_94ae_659e860c1635.slice/crio-2c339673d6b84f5dac9bc8a17fdaf64ff80b2ba24a4a27561724e20ce63ee986 WatchSource:0}: Error finding container 2c339673d6b84f5dac9bc8a17fdaf64ff80b2ba24a4a27561724e20ce63ee986: Status 404 returned error can't find the container with id 2c339673d6b84f5dac9bc8a17fdaf64ff80b2ba24a4a27561724e20ce63ee986 Sep 10 17:49:13.780668 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:49:13.780654 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8be19ca_3248_451c_bc93_b4eebb5e38bd.slice/crio-7e545d2442f48cd980a52254c12a355a46f4f4f14886b53d54b99bc6a30d2c1f WatchSource:0}: Error finding container 7e545d2442f48cd980a52254c12a355a46f4f4f14886b53d54b99bc6a30d2c1f: Status 404 returned error can't find the container with id 7e545d2442f48cd980a52254c12a355a46f4f4f14886b53d54b99bc6a30d2c1f Sep 10 17:49:13.864161 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.864129 2565 generic.go:356] "Generic (PLEG): container finished" podID="ec9e0601-acc4-4456-bfc9-bde2d819f0b6" containerID="d0ea60bb7a2604c09e3d1f8e7906fcb02c8513a462af5375217fd2a16797afad" exitCode=0 Sep 10 17:49:13.864542 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.864175 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:49:13.864542 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.864215 2565 scope.go:117] "RemoveContainer" containerID="d0ea60bb7a2604c09e3d1f8e7906fcb02c8513a462af5375217fd2a16797afad" Sep 10 17:49:13.865860 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.865826 2565 generic.go:356] "Generic (PLEG): container finished" podID="58aa3a28-d38a-4b32-8610-33aeabd91934" containerID="b9c02ade3aa4705cca71e9edb4111c41486f4623b3bd1340a381de69fe73cbfb" exitCode=0 Sep 10 17:49:13.865978 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.865868 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:49:13.866046 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.865982 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mqsdm\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="ec9e0601-acc4-4456-bfc9-bde2d819f0b6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:49:13.867471 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.867444 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="58aa3a28-d38a-4b32-8610-33aeabd91934" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:49:13.868521 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.868461 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" event={"ID":"21fa1895-2079-45ea-bf65-78a26648b98b","Type":"ContainerStarted","Data":"5e22b87a6f4b71694695c4e73df5ac6a80f68d53666d9314fd93d9d72bd88295"} Sep 10 17:49:13.868627 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.868530 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" event={"ID":"21fa1895-2079-45ea-bf65-78a26648b98b","Type":"ContainerStarted","Data":"16e0ea7a682329459fd899318f34723379a67877395099ca355822defaf9a7b9"} Sep 10 17:49:13.868627 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.868559 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" Sep 10 17:49:13.869013 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.868987 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mqsdm\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="ec9e0601-acc4-4456-bfc9-bde2d819f0b6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:49:13.870506 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.870479 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="58aa3a28-d38a-4b32-8610-33aeabd91934" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:49:13.870967 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.870867 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qbl8m" event={"ID":"b8be19ca-3248-451c-bc93-b4eebb5e38bd","Type":"ContainerStarted","Data":"d57c34b05b709fa956164c68dbd91d0d071dce68ffe3569bef543393e6fe6de1"} Sep 10 17:49:13.870967 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.870932 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qbl8m" event={"ID":"b8be19ca-3248-451c-bc93-b4eebb5e38bd","Type":"ContainerStarted","Data":"7e545d2442f48cd980a52254c12a355a46f4f4f14886b53d54b99bc6a30d2c1f"} Sep 10 17:49:13.871163 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.871145 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qbl8m" Sep 10 17:49:13.873182 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.873148 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" event={"ID":"0ebf63a2-b8ce-4cb5-94ae-659e860c1635","Type":"ContainerStarted","Data":"4d49f4d7ebec48998697b4efc5c91e91f0315aa553ad0b7ae1584a9f42466a86"} Sep 10 17:49:13.873269 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.873192 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" event={"ID":"0ebf63a2-b8ce-4cb5-94ae-659e860c1635","Type":"ContainerStarted","Data":"2c339673d6b84f5dac9bc8a17fdaf64ff80b2ba24a4a27561724e20ce63ee986"} Sep 10 17:49:13.873269 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.873230 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" Sep 10 17:49:13.873353 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.873335 2565 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" container="manager" Sep 10 17:49:13.874968 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.874788 2565 scope.go:117] "RemoveContainer" containerID="d0ea60bb7a2604c09e3d1f8e7906fcb02c8513a462af5375217fd2a16797afad" Sep 10 17:49:13.875167 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.875139 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0ea60bb7a2604c09e3d1f8e7906fcb02c8513a462af5375217fd2a16797afad\": container with ID starting with d0ea60bb7a2604c09e3d1f8e7906fcb02c8513a462af5375217fd2a16797afad not found: ID does not exist" containerID="d0ea60bb7a2604c09e3d1f8e7906fcb02c8513a462af5375217fd2a16797afad" Sep 10 17:49:13.875262 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.875177 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0ea60bb7a2604c09e3d1f8e7906fcb02c8513a462af5375217fd2a16797afad"} err="failed to get container status \"d0ea60bb7a2604c09e3d1f8e7906fcb02c8513a462af5375217fd2a16797afad\": rpc error: code = NotFound desc = could not find container \"d0ea60bb7a2604c09e3d1f8e7906fcb02c8513a462af5375217fd2a16797afad\": container with ID starting with d0ea60bb7a2604c09e3d1f8e7906fcb02c8513a462af5375217fd2a16797afad not found: ID does not exist" Sep 10 17:49:13.875262 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.875199 2565 scope.go:117] "RemoveContainer" containerID="b9c02ade3aa4705cca71e9edb4111c41486f4623b3bd1340a381de69fe73cbfb" Sep 10 17:49:13.885385 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.885369 2565 scope.go:117] "RemoveContainer" containerID="b9c02ade3aa4705cca71e9edb4111c41486f4623b3bd1340a381de69fe73cbfb" Sep 10 17:49:13.885649 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.885629 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9c02ade3aa4705cca71e9edb4111c41486f4623b3bd1340a381de69fe73cbfb\": container with ID starting with b9c02ade3aa4705cca71e9edb4111c41486f4623b3bd1340a381de69fe73cbfb not found: ID does not exist" containerID="b9c02ade3aa4705cca71e9edb4111c41486f4623b3bd1340a381de69fe73cbfb" Sep 10 17:49:13.885698 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.885657 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c02ade3aa4705cca71e9edb4111c41486f4623b3bd1340a381de69fe73cbfb"} err="failed to get container status \"b9c02ade3aa4705cca71e9edb4111c41486f4623b3bd1340a381de69fe73cbfb\": rpc error: code = NotFound desc = could not find container \"b9c02ade3aa4705cca71e9edb4111c41486f4623b3bd1340a381de69fe73cbfb\": container with ID starting with b9c02ade3aa4705cca71e9edb4111c41486f4623b3bd1340a381de69fe73cbfb not found: ID does not exist" Sep 10 17:49:13.900972 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.900933 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" podStartSLOduration=0.900921223 podStartE2EDuration="900.921223ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:13.89930831 +0000 UTC m=+557.963719040" watchObservedRunningTime="2026-09-10 17:49:13.900921223 +0000 UTC m=+557.965331931" Sep 10 17:49:13.902000 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.901980 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mqsdm\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="ec9e0601-acc4-4456-bfc9-bde2d819f0b6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:49:13.903920 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.903903 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="58aa3a28-d38a-4b32-8610-33aeabd91934" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:49:13.905129 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.905112 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wckq2\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="58aa3a28-d38a-4b32-8610-33aeabd91934" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wckq2" Sep 10 17:49:13.925649 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.925605 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qbl8m" podStartSLOduration=0.925593208 podStartE2EDuration="925.593208ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:13.924494726 +0000 UTC m=+557.988905460" watchObservedRunningTime="2026-09-10 17:49:13.925593208 +0000 UTC m=+557.990003917" Sep 10 17:49:13.943963 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.943938 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mqsdm\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="ec9e0601-acc4-4456-bfc9-bde2d819f0b6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mqsdm" Sep 10 17:49:13.944068 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:13.943928 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" podStartSLOduration=0.943915651 podStartE2EDuration="943.915651ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:13.942427327 +0000 UTC m=+558.006838041" watchObservedRunningTime="2026-09-10 17:49:13.943915651 +0000 UTC m=+558.008326362" Sep 10 17:49:13.945311 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:13.945291 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-4cnxq\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:14.437906 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:14.437859 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58aa3a28-d38a-4b32-8610-33aeabd91934" path="/var/lib/kubelet/pods/58aa3a28-d38a-4b32-8610-33aeabd91934/volumes" Sep 10 17:49:14.438213 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:14.438198 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec9e0601-acc4-4456-bfc9-bde2d819f0b6" path="/var/lib/kubelet/pods/ec9e0601-acc4-4456-bfc9-bde2d819f0b6/volumes" Sep 10 17:49:14.881256 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:14.881223 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" podUID="2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43" containerName="manager" containerID="cri-o://c10e868fd58ac64f3729f65e1db742710778ecae50934f2718aff06c2d7a1d8f" gracePeriod=2 Sep 10 17:49:15.122874 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:15.122851 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:15.217947 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:15.217913 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43-extensions-socket-volume\") pod \"2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43\" (UID: \"2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43\") " Sep 10 17:49:15.218105 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:15.217968 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vnrs\" (UniqueName: \"kubernetes.io/projected/2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43-kube-api-access-9vnrs\") pod \"2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43\" (UID: \"2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43\") " Sep 10 17:49:15.218336 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:15.218317 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43" (UID: "2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:49:15.219997 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:15.219971 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43-kube-api-access-9vnrs" (OuterVolumeSpecName: "kube-api-access-9vnrs") pod "2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43" (UID: "2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43"). InnerVolumeSpecName "kube-api-access-9vnrs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:49:15.319249 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:15.319219 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43-extensions-socket-volume\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:49:15.319249 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:15.319246 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vnrs\" (UniqueName: \"kubernetes.io/projected/2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43-kube-api-access-9vnrs\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:49:15.353956 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:15.353930 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:15.464298 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:15.464256 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:15.887225 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:15.887188 2565 generic.go:356] "Generic (PLEG): container finished" podID="2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43" containerID="c10e868fd58ac64f3729f65e1db742710778ecae50934f2718aff06c2d7a1d8f" exitCode=0 Sep 10 17:49:15.887672 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:15.887231 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:15.887672 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:15.887289 2565 scope.go:117] "RemoveContainer" containerID="c10e868fd58ac64f3729f65e1db742710778ecae50934f2718aff06c2d7a1d8f" Sep 10 17:49:15.889355 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:15.889332 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-4cnxq\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:15.895845 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:15.895831 2565 scope.go:117] "RemoveContainer" containerID="c10e868fd58ac64f3729f65e1db742710778ecae50934f2718aff06c2d7a1d8f" Sep 10 17:49:15.896119 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:15.896102 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c10e868fd58ac64f3729f65e1db742710778ecae50934f2718aff06c2d7a1d8f\": container with ID starting with c10e868fd58ac64f3729f65e1db742710778ecae50934f2718aff06c2d7a1d8f not found: ID does not exist" containerID="c10e868fd58ac64f3729f65e1db742710778ecae50934f2718aff06c2d7a1d8f" Sep 10 17:49:15.896179 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:15.896125 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c10e868fd58ac64f3729f65e1db742710778ecae50934f2718aff06c2d7a1d8f"} err="failed to get container status \"c10e868fd58ac64f3729f65e1db742710778ecae50934f2718aff06c2d7a1d8f\": rpc error: code = NotFound desc = could not find container \"c10e868fd58ac64f3729f65e1db742710778ecae50934f2718aff06c2d7a1d8f\": container with ID starting with c10e868fd58ac64f3729f65e1db742710778ecae50934f2718aff06c2d7a1d8f not found: ID does not exist" Sep 10 17:49:15.896789 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:15.896769 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-4cnxq\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:16.435570 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:16.435533 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-4cnxq\" is forbidden: User \"system:node:ip-10-0-130-84.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-84.ec2.internal' and this object" podUID="2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-4cnxq" Sep 10 17:49:16.437972 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:16.437942 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43" path="/var/lib/kubelet/pods/2a2ee932-49d2-4cb3-a263-9bf3cc8dbb43/volumes" Sep 10 17:49:17.934013 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:17.933974 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:18.036154 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:18.036123 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:21.882597 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:21.882565 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:23.325576 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:23.325545 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:23.379558 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:23.379529 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:23.727039 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:23.727010 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:23.789369 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:23.789339 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:24.883739 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:24.883700 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-qbl8m" Sep 10 17:49:24.884188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:24.883775 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" Sep 10 17:49:24.884188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:24.883931 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" Sep 10 17:49:24.950141 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:24.950110 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf"] Sep 10 17:49:24.950336 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:24.950309 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" podUID="0ebf63a2-b8ce-4cb5-94ae-659e860c1635" containerName="manager" containerID="cri-o://4d49f4d7ebec48998697b4efc5c91e91f0315aa553ad0b7ae1584a9f42466a86" gracePeriod=10 Sep 10 17:49:25.188461 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.188439 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" Sep 10 17:49:25.301130 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.301103 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgnjt\" (UniqueName: \"kubernetes.io/projected/0ebf63a2-b8ce-4cb5-94ae-659e860c1635-kube-api-access-vgnjt\") pod \"0ebf63a2-b8ce-4cb5-94ae-659e860c1635\" (UID: \"0ebf63a2-b8ce-4cb5-94ae-659e860c1635\") " Sep 10 17:49:25.301294 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.301148 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ebf63a2-b8ce-4cb5-94ae-659e860c1635-extensions-socket-volume\") pod \"0ebf63a2-b8ce-4cb5-94ae-659e860c1635\" (UID: \"0ebf63a2-b8ce-4cb5-94ae-659e860c1635\") " Sep 10 17:49:25.301516 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.301490 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ebf63a2-b8ce-4cb5-94ae-659e860c1635-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0ebf63a2-b8ce-4cb5-94ae-659e860c1635" (UID: "0ebf63a2-b8ce-4cb5-94ae-659e860c1635"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:49:25.302981 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.302950 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ebf63a2-b8ce-4cb5-94ae-659e860c1635-kube-api-access-vgnjt" (OuterVolumeSpecName: "kube-api-access-vgnjt") pod "0ebf63a2-b8ce-4cb5-94ae-659e860c1635" (UID: "0ebf63a2-b8ce-4cb5-94ae-659e860c1635"). InnerVolumeSpecName "kube-api-access-vgnjt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:49:25.402659 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.402629 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgnjt\" (UniqueName: \"kubernetes.io/projected/0ebf63a2-b8ce-4cb5-94ae-659e860c1635-kube-api-access-vgnjt\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:49:25.402659 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.402653 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ebf63a2-b8ce-4cb5-94ae-659e860c1635-extensions-socket-volume\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:49:25.408448 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.408426 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd"] Sep 10 17:49:25.408769 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.408757 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ebf63a2-b8ce-4cb5-94ae-659e860c1635" containerName="manager" Sep 10 17:49:25.408769 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.408770 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ebf63a2-b8ce-4cb5-94ae-659e860c1635" containerName="manager" Sep 10 17:49:25.408859 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.408835 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ebf63a2-b8ce-4cb5-94ae-659e860c1635" containerName="manager" Sep 10 17:49:25.411818 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.411804 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd" Sep 10 17:49:25.430447 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.430389 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd"] Sep 10 17:49:25.503680 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.503655 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/456cdb24-8990-4e20-95f5-353a0c15ea92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tp9cd\" (UID: \"456cdb24-8990-4e20-95f5-353a0c15ea92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd" Sep 10 17:49:25.503813 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.503782 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7bjt\" (UniqueName: \"kubernetes.io/projected/456cdb24-8990-4e20-95f5-353a0c15ea92-kube-api-access-d7bjt\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tp9cd\" (UID: \"456cdb24-8990-4e20-95f5-353a0c15ea92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd" Sep 10 17:49:25.604342 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.604318 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/456cdb24-8990-4e20-95f5-353a0c15ea92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tp9cd\" (UID: \"456cdb24-8990-4e20-95f5-353a0c15ea92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd" Sep 10 17:49:25.604445 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.604379 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7bjt\" (UniqueName: \"kubernetes.io/projected/456cdb24-8990-4e20-95f5-353a0c15ea92-kube-api-access-d7bjt\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tp9cd\" (UID: \"456cdb24-8990-4e20-95f5-353a0c15ea92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd" Sep 10 17:49:25.604648 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.604631 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/456cdb24-8990-4e20-95f5-353a0c15ea92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tp9cd\" (UID: \"456cdb24-8990-4e20-95f5-353a0c15ea92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd" Sep 10 17:49:25.622203 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.622180 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7bjt\" (UniqueName: \"kubernetes.io/projected/456cdb24-8990-4e20-95f5-353a0c15ea92-kube-api-access-d7bjt\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tp9cd\" (UID: \"456cdb24-8990-4e20-95f5-353a0c15ea92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd" Sep 10 17:49:25.720922 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.720864 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd" Sep 10 17:49:25.943859 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.943824 2565 generic.go:356] "Generic (PLEG): container finished" podID="0ebf63a2-b8ce-4cb5-94ae-659e860c1635" containerID="4d49f4d7ebec48998697b4efc5c91e91f0315aa553ad0b7ae1584a9f42466a86" exitCode=0 Sep 10 17:49:25.944282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.943867 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" Sep 10 17:49:25.944282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.943878 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" event={"ID":"0ebf63a2-b8ce-4cb5-94ae-659e860c1635","Type":"ContainerDied","Data":"4d49f4d7ebec48998697b4efc5c91e91f0315aa553ad0b7ae1584a9f42466a86"} Sep 10 17:49:25.944282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.943948 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf" event={"ID":"0ebf63a2-b8ce-4cb5-94ae-659e860c1635","Type":"ContainerDied","Data":"2c339673d6b84f5dac9bc8a17fdaf64ff80b2ba24a4a27561724e20ce63ee986"} Sep 10 17:49:25.944282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.943980 2565 scope.go:117] "RemoveContainer" containerID="4d49f4d7ebec48998697b4efc5c91e91f0315aa553ad0b7ae1584a9f42466a86" Sep 10 17:49:25.952817 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.952633 2565 scope.go:117] "RemoveContainer" containerID="4d49f4d7ebec48998697b4efc5c91e91f0315aa553ad0b7ae1584a9f42466a86" Sep 10 17:49:25.952984 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:25.952966 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d49f4d7ebec48998697b4efc5c91e91f0315aa553ad0b7ae1584a9f42466a86\": container with ID starting with 4d49f4d7ebec48998697b4efc5c91e91f0315aa553ad0b7ae1584a9f42466a86 not found: ID does not exist" containerID="4d49f4d7ebec48998697b4efc5c91e91f0315aa553ad0b7ae1584a9f42466a86" Sep 10 17:49:25.953024 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.952993 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d49f4d7ebec48998697b4efc5c91e91f0315aa553ad0b7ae1584a9f42466a86"} err="failed to get container status \"4d49f4d7ebec48998697b4efc5c91e91f0315aa553ad0b7ae1584a9f42466a86\": rpc error: code = NotFound desc = could not find container \"4d49f4d7ebec48998697b4efc5c91e91f0315aa553ad0b7ae1584a9f42466a86\": container with ID starting with 4d49f4d7ebec48998697b4efc5c91e91f0315aa553ad0b7ae1584a9f42466a86 not found: ID does not exist" Sep 10 17:49:25.968062 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.968040 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf"] Sep 10 17:49:25.976526 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.976463 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-76jjf"] Sep 10 17:49:25.994108 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:25.994082 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd"] Sep 10 17:49:25.995410 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:49:25.995388 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod456cdb24_8990_4e20_95f5_353a0c15ea92.slice/crio-62f5de2c494bde1c9b46788782dd1e2a352e7acbdc6ffc9f75b11aa41ee06377 WatchSource:0}: Error finding container 62f5de2c494bde1c9b46788782dd1e2a352e7acbdc6ffc9f75b11aa41ee06377: Status 404 returned error can't find the container with id 62f5de2c494bde1c9b46788782dd1e2a352e7acbdc6ffc9f75b11aa41ee06377 Sep 10 17:49:26.437001 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:26.436972 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ebf63a2-b8ce-4cb5-94ae-659e860c1635" path="/var/lib/kubelet/pods/0ebf63a2-b8ce-4cb5-94ae-659e860c1635/volumes" Sep 10 17:49:26.951245 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:26.951207 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd" event={"ID":"456cdb24-8990-4e20-95f5-353a0c15ea92","Type":"ContainerStarted","Data":"3b2d81e5b1a068d4319613a51ba11af132bc117ebff8ebc4bfbf5fb3919275cf"} Sep 10 17:49:26.951804 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:26.951778 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd" Sep 10 17:49:26.951947 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:26.951816 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd" event={"ID":"456cdb24-8990-4e20-95f5-353a0c15ea92","Type":"ContainerStarted","Data":"62f5de2c494bde1c9b46788782dd1e2a352e7acbdc6ffc9f75b11aa41ee06377"} Sep 10 17:49:26.971092 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:26.971051 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd" podStartSLOduration=1.971035003 podStartE2EDuration="1.971035003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:26.969656827 +0000 UTC m=+571.034067559" watchObservedRunningTime="2026-09-10 17:49:26.971035003 +0000 UTC m=+571.035445713" Sep 10 17:49:28.057851 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:28.057820 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:28.154031 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:28.154002 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:35.067004 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:35.066966 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:35.168607 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:35.168580 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:37.958706 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:37.958676 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp9cd" Sep 10 17:49:38.021134 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:38.021094 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct"] Sep 10 17:49:38.021338 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:38.021308 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" podUID="21fa1895-2079-45ea-bf65-78a26648b98b" containerName="manager" containerID="cri-o://5e22b87a6f4b71694695c4e73df5ac6a80f68d53666d9314fd93d9d72bd88295" gracePeriod=10 Sep 10 17:49:38.813401 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:38.813374 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" Sep 10 17:49:38.907542 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:38.907486 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21fa1895-2079-45ea-bf65-78a26648b98b-extensions-socket-volume\") pod \"21fa1895-2079-45ea-bf65-78a26648b98b\" (UID: \"21fa1895-2079-45ea-bf65-78a26648b98b\") " Sep 10 17:49:38.907542 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:38.907518 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xv287\" (UniqueName: \"kubernetes.io/projected/21fa1895-2079-45ea-bf65-78a26648b98b-kube-api-access-xv287\") pod \"21fa1895-2079-45ea-bf65-78a26648b98b\" (UID: \"21fa1895-2079-45ea-bf65-78a26648b98b\") " Sep 10 17:49:38.907840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:38.907818 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21fa1895-2079-45ea-bf65-78a26648b98b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "21fa1895-2079-45ea-bf65-78a26648b98b" (UID: "21fa1895-2079-45ea-bf65-78a26648b98b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 17:49:38.909505 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:38.909475 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21fa1895-2079-45ea-bf65-78a26648b98b-kube-api-access-xv287" (OuterVolumeSpecName: "kube-api-access-xv287") pod "21fa1895-2079-45ea-bf65-78a26648b98b" (UID: "21fa1895-2079-45ea-bf65-78a26648b98b"). InnerVolumeSpecName "kube-api-access-xv287". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:49:39.008419 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:39.008397 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21fa1895-2079-45ea-bf65-78a26648b98b-extensions-socket-volume\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:49:39.008419 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:39.008419 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv287\" (UniqueName: \"kubernetes.io/projected/21fa1895-2079-45ea-bf65-78a26648b98b-kube-api-access-xv287\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:49:39.014062 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:39.014034 2565 generic.go:356] "Generic (PLEG): container finished" podID="21fa1895-2079-45ea-bf65-78a26648b98b" containerID="5e22b87a6f4b71694695c4e73df5ac6a80f68d53666d9314fd93d9d72bd88295" exitCode=0 Sep 10 17:49:39.014164 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:39.014070 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" Sep 10 17:49:39.014164 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:39.014096 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" event={"ID":"21fa1895-2079-45ea-bf65-78a26648b98b","Type":"ContainerDied","Data":"5e22b87a6f4b71694695c4e73df5ac6a80f68d53666d9314fd93d9d72bd88295"} Sep 10 17:49:39.014164 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:39.014145 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct" event={"ID":"21fa1895-2079-45ea-bf65-78a26648b98b","Type":"ContainerDied","Data":"16e0ea7a682329459fd899318f34723379a67877395099ca355822defaf9a7b9"} Sep 10 17:49:39.014264 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:39.014174 2565 scope.go:117] "RemoveContainer" containerID="5e22b87a6f4b71694695c4e73df5ac6a80f68d53666d9314fd93d9d72bd88295" Sep 10 17:49:39.023126 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:39.022955 2565 scope.go:117] "RemoveContainer" containerID="5e22b87a6f4b71694695c4e73df5ac6a80f68d53666d9314fd93d9d72bd88295" Sep 10 17:49:39.023218 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:49:39.023200 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e22b87a6f4b71694695c4e73df5ac6a80f68d53666d9314fd93d9d72bd88295\": container with ID starting with 5e22b87a6f4b71694695c4e73df5ac6a80f68d53666d9314fd93d9d72bd88295 not found: ID does not exist" containerID="5e22b87a6f4b71694695c4e73df5ac6a80f68d53666d9314fd93d9d72bd88295" Sep 10 17:49:39.023254 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:39.023226 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e22b87a6f4b71694695c4e73df5ac6a80f68d53666d9314fd93d9d72bd88295"} err="failed to get container status \"5e22b87a6f4b71694695c4e73df5ac6a80f68d53666d9314fd93d9d72bd88295\": rpc error: code = NotFound desc = could not find container \"5e22b87a6f4b71694695c4e73df5ac6a80f68d53666d9314fd93d9d72bd88295\": container with ID starting with 5e22b87a6f4b71694695c4e73df5ac6a80f68d53666d9314fd93d9d72bd88295 not found: ID does not exist" Sep 10 17:49:39.036225 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:39.036206 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct"] Sep 10 17:49:39.042049 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:39.042029 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-wrpct"] Sep 10 17:49:40.437821 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:40.437788 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21fa1895-2079-45ea-bf65-78a26648b98b" path="/var/lib/kubelet/pods/21fa1895-2079-45ea-bf65-78a26648b98b/volumes" Sep 10 17:49:41.453135 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:41.453108 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:41.855600 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:41.855526 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:44.215346 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:44.215306 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:56.373188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:56.373162 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g2pbs_f558fde8-d070-4538-bb97-19bb59832b85/ovn-acl-logging/0.log" Sep 10 17:49:56.375299 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:56.375276 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g2pbs_f558fde8-d070-4538-bb97-19bb59832b85/ovn-acl-logging/0.log" Sep 10 17:49:57.788118 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:57.788086 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-927h6"] Sep 10 17:49:57.788472 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:57.788432 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21fa1895-2079-45ea-bf65-78a26648b98b" containerName="manager" Sep 10 17:49:57.788472 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:57.788443 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="21fa1895-2079-45ea-bf65-78a26648b98b" containerName="manager" Sep 10 17:49:57.788543 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:57.788512 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="21fa1895-2079-45ea-bf65-78a26648b98b" containerName="manager" Sep 10 17:49:57.791316 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:57.791288 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:57.792773 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:57.792757 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-927h6" Sep 10 17:49:57.794509 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:57.794493 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 10 17:49:57.801142 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:57.801114 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-927h6"] Sep 10 17:49:57.889587 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:57.889520 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-927h6"] Sep 10 17:49:57.896985 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:57.896957 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:57.947554 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:57.947526 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l8hp\" (UniqueName: \"kubernetes.io/projected/4b300903-103c-4c4f-875d-b221448ae26b-kube-api-access-8l8hp\") pod \"limitador-limitador-8588749977-927h6\" (UID: \"4b300903-103c-4c4f-875d-b221448ae26b\") " pod="kuadrant-system/limitador-limitador-8588749977-927h6" Sep 10 17:49:57.947695 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:57.947570 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4b300903-103c-4c4f-875d-b221448ae26b-config-file\") pod \"limitador-limitador-8588749977-927h6\" (UID: \"4b300903-103c-4c4f-875d-b221448ae26b\") " pod="kuadrant-system/limitador-limitador-8588749977-927h6" Sep 10 17:49:57.956984 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:57.956941 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:58.048764 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:58.048687 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l8hp\" (UniqueName: \"kubernetes.io/projected/4b300903-103c-4c4f-875d-b221448ae26b-kube-api-access-8l8hp\") pod \"limitador-limitador-8588749977-927h6\" (UID: \"4b300903-103c-4c4f-875d-b221448ae26b\") " pod="kuadrant-system/limitador-limitador-8588749977-927h6" Sep 10 17:49:58.048764 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:58.048728 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4b300903-103c-4c4f-875d-b221448ae26b-config-file\") pod \"limitador-limitador-8588749977-927h6\" (UID: \"4b300903-103c-4c4f-875d-b221448ae26b\") " pod="kuadrant-system/limitador-limitador-8588749977-927h6" Sep 10 17:49:58.049395 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:58.049376 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4b300903-103c-4c4f-875d-b221448ae26b-config-file\") pod \"limitador-limitador-8588749977-927h6\" (UID: \"4b300903-103c-4c4f-875d-b221448ae26b\") " pod="kuadrant-system/limitador-limitador-8588749977-927h6" Sep 10 17:49:58.056325 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:58.056297 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l8hp\" (UniqueName: \"kubernetes.io/projected/4b300903-103c-4c4f-875d-b221448ae26b-kube-api-access-8l8hp\") pod \"limitador-limitador-8588749977-927h6\" (UID: \"4b300903-103c-4c4f-875d-b221448ae26b\") " pod="kuadrant-system/limitador-limitador-8588749977-927h6" Sep 10 17:49:58.104393 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:58.104365 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-927h6" Sep 10 17:49:58.373146 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:58.373117 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-927h6"] Sep 10 17:49:58.375414 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:49:58.375389 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b300903_103c_4c4f_875d_b221448ae26b.slice/crio-67fccd43d92aea70fae5e5ef90abc17018ac0ec9359e1a44aabce833fdf13044 WatchSource:0}: Error finding container 67fccd43d92aea70fae5e5ef90abc17018ac0ec9359e1a44aabce833fdf13044: Status 404 returned error can't find the container with id 67fccd43d92aea70fae5e5ef90abc17018ac0ec9359e1a44aabce833fdf13044 Sep 10 17:49:58.622481 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:58.622398 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:58.703592 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:58.703565 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:49:59.111786 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:49:59.111738 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-927h6" event={"ID":"4b300903-103c-4c4f-875d-b221448ae26b","Type":"ContainerStarted","Data":"67fccd43d92aea70fae5e5ef90abc17018ac0ec9359e1a44aabce833fdf13044"} Sep 10 17:50:00.117690 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:00.117654 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-927h6" event={"ID":"4b300903-103c-4c4f-875d-b221448ae26b","Type":"ContainerStarted","Data":"8eefee0d6afb194afb89965d3793d779737cffd1a8c528f6e64522a49ecc4cd5"} Sep 10 17:50:00.118149 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:00.117718 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-927h6" Sep 10 17:50:00.138824 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:00.138780 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-927h6" podStartSLOduration=2.325134169 podStartE2EDuration="3.138765939s" totalImagesPullingTime="813.63177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:50:00.138650647 +0000 UTC m=+604.203061375" watchObservedRunningTime="2026-09-10 17:50:00.138765939 +0000 UTC m=+604.203176651" Sep 10 17:50:08.214516 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:08.214480 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:11.123728 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:11.123691 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-927h6" Sep 10 17:50:12.810967 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:12.810936 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-927h6"] Sep 10 17:50:12.811326 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:12.811095 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-927h6" podUID="4b300903-103c-4c4f-875d-b221448ae26b" containerName="limitador" containerID="cri-o://8eefee0d6afb194afb89965d3793d779737cffd1a8c528f6e64522a49ecc4cd5" gracePeriod=30 Sep 10 17:50:13.176545 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:13.176510 2565 generic.go:356] "Generic (PLEG): container finished" podID="4b300903-103c-4c4f-875d-b221448ae26b" containerID="8eefee0d6afb194afb89965d3793d779737cffd1a8c528f6e64522a49ecc4cd5" exitCode=0 Sep 10 17:50:13.176690 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:13.176581 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-927h6" event={"ID":"4b300903-103c-4c4f-875d-b221448ae26b","Type":"ContainerDied","Data":"8eefee0d6afb194afb89965d3793d779737cffd1a8c528f6e64522a49ecc4cd5"} Sep 10 17:50:13.747203 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:13.747179 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-927h6" Sep 10 17:50:13.878049 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:13.877969 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l8hp\" (UniqueName: \"kubernetes.io/projected/4b300903-103c-4c4f-875d-b221448ae26b-kube-api-access-8l8hp\") pod \"4b300903-103c-4c4f-875d-b221448ae26b\" (UID: \"4b300903-103c-4c4f-875d-b221448ae26b\") " Sep 10 17:50:13.878406 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:13.878051 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4b300903-103c-4c4f-875d-b221448ae26b-config-file\") pod \"4b300903-103c-4c4f-875d-b221448ae26b\" (UID: \"4b300903-103c-4c4f-875d-b221448ae26b\") " Sep 10 17:50:13.878455 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:13.878416 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b300903-103c-4c4f-875d-b221448ae26b-config-file" (OuterVolumeSpecName: "config-file") pod "4b300903-103c-4c4f-875d-b221448ae26b" (UID: "4b300903-103c-4c4f-875d-b221448ae26b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 17:50:13.880076 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:13.880055 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b300903-103c-4c4f-875d-b221448ae26b-kube-api-access-8l8hp" (OuterVolumeSpecName: "kube-api-access-8l8hp") pod "4b300903-103c-4c4f-875d-b221448ae26b" (UID: "4b300903-103c-4c4f-875d-b221448ae26b"). InnerVolumeSpecName "kube-api-access-8l8hp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:50:13.979328 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:13.979292 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l8hp\" (UniqueName: \"kubernetes.io/projected/4b300903-103c-4c4f-875d-b221448ae26b-kube-api-access-8l8hp\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:50:13.979328 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:13.979323 2565 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4b300903-103c-4c4f-875d-b221448ae26b-config-file\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:50:14.182903 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:14.182859 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-927h6" Sep 10 17:50:14.182903 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:14.182865 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-927h6" event={"ID":"4b300903-103c-4c4f-875d-b221448ae26b","Type":"ContainerDied","Data":"67fccd43d92aea70fae5e5ef90abc17018ac0ec9359e1a44aabce833fdf13044"} Sep 10 17:50:14.183109 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:14.182947 2565 scope.go:117] "RemoveContainer" containerID="8eefee0d6afb194afb89965d3793d779737cffd1a8c528f6e64522a49ecc4cd5" Sep 10 17:50:14.207651 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:14.207621 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-927h6"] Sep 10 17:50:14.210723 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:14.210697 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-927h6"] Sep 10 17:50:14.437257 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:14.437176 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b300903-103c-4c4f-875d-b221448ae26b" path="/var/lib/kubelet/pods/4b300903-103c-4c4f-875d-b221448ae26b/volumes" Sep 10 17:50:15.257078 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.257048 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:15.356042 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.356011 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:15.742310 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.742276 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-787tw"] Sep 10 17:50:15.742796 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.742782 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b300903-103c-4c4f-875d-b221448ae26b" containerName="limitador" Sep 10 17:50:15.742846 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.742799 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b300903-103c-4c4f-875d-b221448ae26b" containerName="limitador" Sep 10 17:50:15.742919 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.742907 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b300903-103c-4c4f-875d-b221448ae26b" containerName="limitador" Sep 10 17:50:15.747356 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.747334 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-787tw" Sep 10 17:50:15.750037 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.750012 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-mnxl7\"" Sep 10 17:50:15.750037 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.750028 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 17:50:15.750225 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.750063 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 10 17:50:15.750225 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.750012 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 10 17:50:15.751580 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.751533 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-787tw"] Sep 10 17:50:15.767720 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.767693 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:15.895740 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.895701 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbzz7\" (UniqueName: \"kubernetes.io/projected/8627c198-ef2f-4728-aef6-6b64e9592954-kube-api-access-gbzz7\") pod \"postgres-69fb4594cf-787tw\" (UID: \"8627c198-ef2f-4728-aef6-6b64e9592954\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-787tw" Sep 10 17:50:15.895936 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.895768 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8627c198-ef2f-4728-aef6-6b64e9592954-data\") pod \"postgres-69fb4594cf-787tw\" (UID: \"8627c198-ef2f-4728-aef6-6b64e9592954\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-787tw" Sep 10 17:50:15.996605 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.996514 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8627c198-ef2f-4728-aef6-6b64e9592954-data\") pod \"postgres-69fb4594cf-787tw\" (UID: \"8627c198-ef2f-4728-aef6-6b64e9592954\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-787tw" Sep 10 17:50:15.996605 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.996595 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbzz7\" (UniqueName: \"kubernetes.io/projected/8627c198-ef2f-4728-aef6-6b64e9592954-kube-api-access-gbzz7\") pod \"postgres-69fb4594cf-787tw\" (UID: \"8627c198-ef2f-4728-aef6-6b64e9592954\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-787tw" Sep 10 17:50:15.997002 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:15.996979 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8627c198-ef2f-4728-aef6-6b64e9592954-data\") pod \"postgres-69fb4594cf-787tw\" (UID: \"8627c198-ef2f-4728-aef6-6b64e9592954\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-787tw" Sep 10 17:50:16.004494 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:16.004473 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbzz7\" (UniqueName: \"kubernetes.io/projected/8627c198-ef2f-4728-aef6-6b64e9592954-kube-api-access-gbzz7\") pod \"postgres-69fb4594cf-787tw\" (UID: \"8627c198-ef2f-4728-aef6-6b64e9592954\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-787tw" Sep 10 17:50:16.058396 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:16.058368 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-787tw" Sep 10 17:50:16.325399 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:16.325366 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-787tw"] Sep 10 17:50:16.327730 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:50:16.327692 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8627c198_ef2f_4728_aef6_6b64e9592954.slice/crio-55f5ea11787081b9c828ca4393f6c800090af9db5a25f59b8c0a231e376e860b WatchSource:0}: Error finding container 55f5ea11787081b9c828ca4393f6c800090af9db5a25f59b8c0a231e376e860b: Status 404 returned error can't find the container with id 55f5ea11787081b9c828ca4393f6c800090af9db5a25f59b8c0a231e376e860b Sep 10 17:50:17.202430 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:17.202391 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-787tw" event={"ID":"8627c198-ef2f-4728-aef6-6b64e9592954","Type":"ContainerStarted","Data":"55f5ea11787081b9c828ca4393f6c800090af9db5a25f59b8c0a231e376e860b"} Sep 10 17:50:19.214026 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:19.213990 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-787tw" event={"ID":"8627c198-ef2f-4728-aef6-6b64e9592954","Type":"ContainerStarted","Data":"4d7c4a129faac720ff95f9dea560608bd061a251415560321c2504d0e2268643"} Sep 10 17:50:19.214390 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:19.214051 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-787tw" Sep 10 17:50:19.230650 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:19.230503 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-787tw" podStartSLOduration=2.059870779 podStartE2EDuration="4.230483894s" totalImagesPullingTime="2.170613115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:50:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:50:19.22948426 +0000 UTC m=+623.293895006" watchObservedRunningTime="2026-09-10 17:50:19.230483894 +0000 UTC m=+623.294894608" Sep 10 17:50:25.246919 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:25.246880 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-787tw" Sep 10 17:50:25.564612 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:25.564519 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:25.839395 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:25.839310 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:33.245095 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.245062 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:33.318998 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.318961 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bbdd8d44-4rpc7"] Sep 10 17:50:33.326837 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.326814 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:33.329157 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.329130 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 10 17:50:33.329467 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.329446 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 10 17:50:33.329467 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.329462 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8cbcx\"" Sep 10 17:50:33.330373 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.330351 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 10 17:50:33.332641 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.332621 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bbdd8d44-4rpc7"] Sep 10 17:50:33.339241 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.339220 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:33.437513 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.437487 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:33.448531 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.448508 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-metrics-certs\") pod \"maas-controller-7bbdd8d44-4rpc7\" (UID: \"84502821-cbcb-4be7-9ba2-e1a39452f938\") " pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:33.448608 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.448546 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-webhook-cert\") pod \"maas-controller-7bbdd8d44-4rpc7\" (UID: \"84502821-cbcb-4be7-9ba2-e1a39452f938\") " pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:33.448608 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.448566 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxs4w\" (UniqueName: \"kubernetes.io/projected/84502821-cbcb-4be7-9ba2-e1a39452f938-kube-api-access-dxs4w\") pod \"maas-controller-7bbdd8d44-4rpc7\" (UID: \"84502821-cbcb-4be7-9ba2-e1a39452f938\") " pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:33.510640 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.510563 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-864f69dd8b-2s62v"] Sep 10 17:50:33.511538 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.511516 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:33.514384 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.514366 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-864f69dd8b-2s62v" Sep 10 17:50:33.524707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.524676 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-864f69dd8b-2s62v"] Sep 10 17:50:33.549239 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.549214 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-webhook-cert\") pod \"maas-controller-7bbdd8d44-4rpc7\" (UID: \"84502821-cbcb-4be7-9ba2-e1a39452f938\") " pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:33.549355 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.549250 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxs4w\" (UniqueName: \"kubernetes.io/projected/84502821-cbcb-4be7-9ba2-e1a39452f938-kube-api-access-dxs4w\") pod \"maas-controller-7bbdd8d44-4rpc7\" (UID: \"84502821-cbcb-4be7-9ba2-e1a39452f938\") " pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:33.549355 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.549272 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/88764722-e6e6-4356-929c-8fb593091a1e-webhook-cert\") pod \"maas-controller-864f69dd8b-2s62v\" (UID: \"88764722-e6e6-4356-929c-8fb593091a1e\") " pod="opendatahub/maas-controller-864f69dd8b-2s62v" Sep 10 17:50:33.549355 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.549349 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88764722-e6e6-4356-929c-8fb593091a1e-metrics-certs\") pod \"maas-controller-864f69dd8b-2s62v\" (UID: \"88764722-e6e6-4356-929c-8fb593091a1e\") " pod="opendatahub/maas-controller-864f69dd8b-2s62v" Sep 10 17:50:33.549524 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:50:33.549370 2565 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 10 17:50:33.549524 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.549374 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-metrics-certs\") pod \"maas-controller-7bbdd8d44-4rpc7\" (UID: \"84502821-cbcb-4be7-9ba2-e1a39452f938\") " pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:33.549524 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:50:33.549430 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-webhook-cert podName:84502821-cbcb-4be7-9ba2-e1a39452f938 nodeName:}" failed. No retries permitted until 2026-09-10 17:50:34.049410957 +0000 UTC m=+638.113821659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-webhook-cert") pod "maas-controller-7bbdd8d44-4rpc7" (UID: "84502821-cbcb-4be7-9ba2-e1a39452f938") : secret "maas-controller-webhook-cert" not found Sep 10 17:50:33.549524 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.549473 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxwkd\" (UniqueName: \"kubernetes.io/projected/88764722-e6e6-4356-929c-8fb593091a1e-kube-api-access-xxwkd\") pod \"maas-controller-864f69dd8b-2s62v\" (UID: \"88764722-e6e6-4356-929c-8fb593091a1e\") " pod="opendatahub/maas-controller-864f69dd8b-2s62v" Sep 10 17:50:33.551613 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.551596 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-metrics-certs\") pod \"maas-controller-7bbdd8d44-4rpc7\" (UID: \"84502821-cbcb-4be7-9ba2-e1a39452f938\") " pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:33.557198 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.557177 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxs4w\" (UniqueName: \"kubernetes.io/projected/84502821-cbcb-4be7-9ba2-e1a39452f938-kube-api-access-dxs4w\") pod \"maas-controller-7bbdd8d44-4rpc7\" (UID: \"84502821-cbcb-4be7-9ba2-e1a39452f938\") " pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:33.650404 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.650376 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/88764722-e6e6-4356-929c-8fb593091a1e-webhook-cert\") pod \"maas-controller-864f69dd8b-2s62v\" (UID: \"88764722-e6e6-4356-929c-8fb593091a1e\") " pod="opendatahub/maas-controller-864f69dd8b-2s62v" Sep 10 17:50:33.650523 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.650437 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88764722-e6e6-4356-929c-8fb593091a1e-metrics-certs\") pod \"maas-controller-864f69dd8b-2s62v\" (UID: \"88764722-e6e6-4356-929c-8fb593091a1e\") " pod="opendatahub/maas-controller-864f69dd8b-2s62v" Sep 10 17:50:33.650523 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.650455 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxwkd\" (UniqueName: \"kubernetes.io/projected/88764722-e6e6-4356-929c-8fb593091a1e-kube-api-access-xxwkd\") pod \"maas-controller-864f69dd8b-2s62v\" (UID: \"88764722-e6e6-4356-929c-8fb593091a1e\") " pod="opendatahub/maas-controller-864f69dd8b-2s62v" Sep 10 17:50:33.652693 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.652669 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88764722-e6e6-4356-929c-8fb593091a1e-metrics-certs\") pod \"maas-controller-864f69dd8b-2s62v\" (UID: \"88764722-e6e6-4356-929c-8fb593091a1e\") " pod="opendatahub/maas-controller-864f69dd8b-2s62v" Sep 10 17:50:33.652858 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.652835 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/88764722-e6e6-4356-929c-8fb593091a1e-webhook-cert\") pod \"maas-controller-864f69dd8b-2s62v\" (UID: \"88764722-e6e6-4356-929c-8fb593091a1e\") " pod="opendatahub/maas-controller-864f69dd8b-2s62v" Sep 10 17:50:33.658490 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.658468 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxwkd\" (UniqueName: \"kubernetes.io/projected/88764722-e6e6-4356-929c-8fb593091a1e-kube-api-access-xxwkd\") pod \"maas-controller-864f69dd8b-2s62v\" (UID: \"88764722-e6e6-4356-929c-8fb593091a1e\") " pod="opendatahub/maas-controller-864f69dd8b-2s62v" Sep 10 17:50:33.685222 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.685198 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:33.825609 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:33.825527 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-864f69dd8b-2s62v" Sep 10 17:50:34.055134 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:34.055098 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-webhook-cert\") pod \"maas-controller-7bbdd8d44-4rpc7\" (UID: \"84502821-cbcb-4be7-9ba2-e1a39452f938\") " pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:34.057367 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:34.057348 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-webhook-cert\") pod \"maas-controller-7bbdd8d44-4rpc7\" (UID: \"84502821-cbcb-4be7-9ba2-e1a39452f938\") " pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:34.095286 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:34.095264 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-864f69dd8b-2s62v"] Sep 10 17:50:34.097232 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:50:34.097207 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88764722_e6e6_4356_929c_8fb593091a1e.slice/crio-666e011ad2e999b8ec20c2afca1b7b64b23703ecf0c2f98a12d24ee3e29737c5 WatchSource:0}: Error finding container 666e011ad2e999b8ec20c2afca1b7b64b23703ecf0c2f98a12d24ee3e29737c5: Status 404 returned error can't find the container with id 666e011ad2e999b8ec20c2afca1b7b64b23703ecf0c2f98a12d24ee3e29737c5 Sep 10 17:50:34.237514 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:34.237480 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:34.287532 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:34.287498 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-864f69dd8b-2s62v" event={"ID":"88764722-e6e6-4356-929c-8fb593091a1e","Type":"ContainerStarted","Data":"666e011ad2e999b8ec20c2afca1b7b64b23703ecf0c2f98a12d24ee3e29737c5"} Sep 10 17:50:34.506779 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:34.506757 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bbdd8d44-4rpc7"] Sep 10 17:50:34.508769 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:50:34.508741 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84502821_cbcb_4be7_9ba2_e1a39452f938.slice/crio-cb28a94038fcda72aff7560a9170d05f793da69832732e3af378fb3eb4e2a5fc WatchSource:0}: Error finding container cb28a94038fcda72aff7560a9170d05f793da69832732e3af378fb3eb4e2a5fc: Status 404 returned error can't find the container with id cb28a94038fcda72aff7560a9170d05f793da69832732e3af378fb3eb4e2a5fc Sep 10 17:50:35.295666 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:35.295630 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" event={"ID":"84502821-cbcb-4be7-9ba2-e1a39452f938","Type":"ContainerStarted","Data":"cb28a94038fcda72aff7560a9170d05f793da69832732e3af378fb3eb4e2a5fc"} Sep 10 17:50:35.502341 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:35.502297 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:36.035345 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:36.035305 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:36.135749 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:36.135719 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:36.302147 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:36.302117 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" event={"ID":"84502821-cbcb-4be7-9ba2-e1a39452f938","Type":"ContainerStarted","Data":"c47a60d117c9c2cfb5a43e224e133b0575c564fe8d16d273142f1fe15c869df6"} Sep 10 17:50:36.302560 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:36.302217 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:36.303657 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:36.303637 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-864f69dd8b-2s62v" event={"ID":"88764722-e6e6-4356-929c-8fb593091a1e","Type":"ContainerStarted","Data":"80f567832f1228dacdf7e61165341543f3ec7c5b34be6ff8eebb1ae1272dd349"} Sep 10 17:50:36.303770 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:36.303744 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-864f69dd8b-2s62v" Sep 10 17:50:36.320387 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:36.320331 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" podStartSLOduration=1.966724331 podStartE2EDuration="3.320315965s" totalImagesPullingTime="1.353591634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:50:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:50:36.318151085 +0000 UTC m=+640.382561814" watchObservedRunningTime="2026-09-10 17:50:36.320315965 +0000 UTC m=+640.384726675" Sep 10 17:50:36.337942 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:36.337894 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-864f69dd8b-2s62v" podStartSLOduration=1.575393144 podStartE2EDuration="3.337868746s" totalImagesPullingTime="1.762475602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:50:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:50:36.335599971 +0000 UTC m=+640.400010700" watchObservedRunningTime="2026-09-10 17:50:36.337868746 +0000 UTC m=+640.402279454" Sep 10 17:50:37.817247 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:37.817210 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:37.891340 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:37.891302 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:42.219554 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:42.219516 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:50:47.309818 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:47.309788 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-864f69dd8b-2s62v" Sep 10 17:50:47.310239 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:47.309948 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:47.373098 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:47.373067 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bbdd8d44-4rpc7"] Sep 10 17:50:47.373898 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:47.373825 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" podUID="84502821-cbcb-4be7-9ba2-e1a39452f938" containerName="manager" containerID="cri-o://c47a60d117c9c2cfb5a43e224e133b0575c564fe8d16d273142f1fe15c869df6" gracePeriod=30 Sep 10 17:50:47.614961 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:47.614941 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:47.669131 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:47.669101 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-metrics-certs\") pod \"84502821-cbcb-4be7-9ba2-e1a39452f938\" (UID: \"84502821-cbcb-4be7-9ba2-e1a39452f938\") " Sep 10 17:50:47.669276 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:47.669167 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-webhook-cert\") pod \"84502821-cbcb-4be7-9ba2-e1a39452f938\" (UID: \"84502821-cbcb-4be7-9ba2-e1a39452f938\") " Sep 10 17:50:47.669276 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:47.669224 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxs4w\" (UniqueName: \"kubernetes.io/projected/84502821-cbcb-4be7-9ba2-e1a39452f938-kube-api-access-dxs4w\") pod \"84502821-cbcb-4be7-9ba2-e1a39452f938\" (UID: \"84502821-cbcb-4be7-9ba2-e1a39452f938\") " Sep 10 17:50:47.671278 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:47.671251 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "84502821-cbcb-4be7-9ba2-e1a39452f938" (UID: "84502821-cbcb-4be7-9ba2-e1a39452f938"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:50:47.671278 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:47.671268 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "84502821-cbcb-4be7-9ba2-e1a39452f938" (UID: "84502821-cbcb-4be7-9ba2-e1a39452f938"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 17:50:47.671414 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:47.671360 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84502821-cbcb-4be7-9ba2-e1a39452f938-kube-api-access-dxs4w" (OuterVolumeSpecName: "kube-api-access-dxs4w") pod "84502821-cbcb-4be7-9ba2-e1a39452f938" (UID: "84502821-cbcb-4be7-9ba2-e1a39452f938"). InnerVolumeSpecName "kube-api-access-dxs4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 17:50:47.770724 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:47.770703 2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-metrics-certs\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:50:47.770724 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:47.770724 2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84502821-cbcb-4be7-9ba2-e1a39452f938-webhook-cert\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:50:47.770882 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:47.770733 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxs4w\" (UniqueName: \"kubernetes.io/projected/84502821-cbcb-4be7-9ba2-e1a39452f938-kube-api-access-dxs4w\") on node \"ip-10-0-130-84.ec2.internal\" DevicePath \"\"" Sep 10 17:50:48.360380 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:48.360345 2565 generic.go:356] "Generic (PLEG): container finished" podID="84502821-cbcb-4be7-9ba2-e1a39452f938" containerID="c47a60d117c9c2cfb5a43e224e133b0575c564fe8d16d273142f1fe15c869df6" exitCode=0 Sep 10 17:50:48.360730 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:48.360390 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" Sep 10 17:50:48.360730 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:48.360431 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" event={"ID":"84502821-cbcb-4be7-9ba2-e1a39452f938","Type":"ContainerDied","Data":"c47a60d117c9c2cfb5a43e224e133b0575c564fe8d16d273142f1fe15c869df6"} Sep 10 17:50:48.360730 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:48.360486 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bbdd8d44-4rpc7" event={"ID":"84502821-cbcb-4be7-9ba2-e1a39452f938","Type":"ContainerDied","Data":"cb28a94038fcda72aff7560a9170d05f793da69832732e3af378fb3eb4e2a5fc"} Sep 10 17:50:48.360730 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:48.360514 2565 scope.go:117] "RemoveContainer" containerID="c47a60d117c9c2cfb5a43e224e133b0575c564fe8d16d273142f1fe15c869df6" Sep 10 17:50:48.369658 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:48.369642 2565 scope.go:117] "RemoveContainer" containerID="c47a60d117c9c2cfb5a43e224e133b0575c564fe8d16d273142f1fe15c869df6" Sep 10 17:50:48.369925 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:50:48.369905 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c47a60d117c9c2cfb5a43e224e133b0575c564fe8d16d273142f1fe15c869df6\": container with ID starting with c47a60d117c9c2cfb5a43e224e133b0575c564fe8d16d273142f1fe15c869df6 not found: ID does not exist" containerID="c47a60d117c9c2cfb5a43e224e133b0575c564fe8d16d273142f1fe15c869df6" Sep 10 17:50:48.370010 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:48.369934 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47a60d117c9c2cfb5a43e224e133b0575c564fe8d16d273142f1fe15c869df6"} err="failed to get container status \"c47a60d117c9c2cfb5a43e224e133b0575c564fe8d16d273142f1fe15c869df6\": rpc error: code = NotFound desc = could not find container \"c47a60d117c9c2cfb5a43e224e133b0575c564fe8d16d273142f1fe15c869df6\": container with ID starting with c47a60d117c9c2cfb5a43e224e133b0575c564fe8d16d273142f1fe15c869df6 not found: ID does not exist" Sep 10 17:50:48.380915 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:48.380872 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bbdd8d44-4rpc7"] Sep 10 17:50:48.386062 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:48.386042 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7bbdd8d44-4rpc7"] Sep 10 17:50:48.437505 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:48.437485 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84502821-cbcb-4be7-9ba2-e1a39452f938" path="/var/lib/kubelet/pods/84502821-cbcb-4be7-9ba2-e1a39452f938/volumes" Sep 10 17:50:53.044972 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:50:53.044940 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:08.674634 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:08.674597 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:09.791278 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:09.791239 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:10.133707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.133681 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:10.238739 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.238686 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:10.251832 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.251802 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5"] Sep 10 17:51:10.252237 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.252222 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84502821-cbcb-4be7-9ba2-e1a39452f938" containerName="manager" Sep 10 17:51:10.252291 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.252238 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="84502821-cbcb-4be7-9ba2-e1a39452f938" containerName="manager" Sep 10 17:51:10.252327 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.252321 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="84502821-cbcb-4be7-9ba2-e1a39452f938" containerName="manager" Sep 10 17:51:10.254937 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.254916 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:10.258227 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.258201 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q8r8w\"" Sep 10 17:51:10.258335 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.258207 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 10 17:51:10.258335 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.258210 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 10 17:51:10.269805 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.269785 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5"] Sep 10 17:51:10.353787 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.353750 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7ae526-4baf-4912-bc21-d71d1a447094-openshift-service-ca\") pod \"maas-api-7f975847c4-9f7c5\" (UID: \"3e7ae526-4baf-4912-bc21-d71d1a447094\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:10.353996 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.353793 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl8g4\" (UniqueName: \"kubernetes.io/projected/3e7ae526-4baf-4912-bc21-d71d1a447094-kube-api-access-tl8g4\") pod \"maas-api-7f975847c4-9f7c5\" (UID: \"3e7ae526-4baf-4912-bc21-d71d1a447094\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:10.353996 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.353822 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7ae526-4baf-4912-bc21-d71d1a447094-maas-api-tls\") pod \"maas-api-7f975847c4-9f7c5\" (UID: \"3e7ae526-4baf-4912-bc21-d71d1a447094\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:10.353996 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.353836 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e7ae526-4baf-4912-bc21-d71d1a447094-metrics-certs\") pod \"maas-api-7f975847c4-9f7c5\" (UID: \"3e7ae526-4baf-4912-bc21-d71d1a447094\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:10.392050 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.391983 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:10.455001 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.454976 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7ae526-4baf-4912-bc21-d71d1a447094-openshift-service-ca\") pod \"maas-api-7f975847c4-9f7c5\" (UID: \"3e7ae526-4baf-4912-bc21-d71d1a447094\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:10.455156 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.455011 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl8g4\" (UniqueName: \"kubernetes.io/projected/3e7ae526-4baf-4912-bc21-d71d1a447094-kube-api-access-tl8g4\") pod \"maas-api-7f975847c4-9f7c5\" (UID: \"3e7ae526-4baf-4912-bc21-d71d1a447094\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:10.455156 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.455039 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7ae526-4baf-4912-bc21-d71d1a447094-maas-api-tls\") pod \"maas-api-7f975847c4-9f7c5\" (UID: \"3e7ae526-4baf-4912-bc21-d71d1a447094\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:10.455156 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.455060 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e7ae526-4baf-4912-bc21-d71d1a447094-metrics-certs\") pod \"maas-api-7f975847c4-9f7c5\" (UID: \"3e7ae526-4baf-4912-bc21-d71d1a447094\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:10.455684 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.455666 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7ae526-4baf-4912-bc21-d71d1a447094-openshift-service-ca\") pod \"maas-api-7f975847c4-9f7c5\" (UID: \"3e7ae526-4baf-4912-bc21-d71d1a447094\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:10.457388 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.457372 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7ae526-4baf-4912-bc21-d71d1a447094-maas-api-tls\") pod \"maas-api-7f975847c4-9f7c5\" (UID: \"3e7ae526-4baf-4912-bc21-d71d1a447094\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:10.457534 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.457515 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e7ae526-4baf-4912-bc21-d71d1a447094-metrics-certs\") pod \"maas-api-7f975847c4-9f7c5\" (UID: \"3e7ae526-4baf-4912-bc21-d71d1a447094\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:10.462245 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.462217 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl8g4\" (UniqueName: \"kubernetes.io/projected/3e7ae526-4baf-4912-bc21-d71d1a447094-kube-api-access-tl8g4\") pod \"maas-api-7f975847c4-9f7c5\" (UID: \"3e7ae526-4baf-4912-bc21-d71d1a447094\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:10.565927 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.565873 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:10.676487 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.676413 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:10.784274 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.784240 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:10.785728 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.785708 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-scxln"] Sep 10 17:51:10.788372 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.788354 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-scxln" Sep 10 17:51:10.790460 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.790436 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-n785n\"" Sep 10 17:51:10.790618 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.790600 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 10 17:51:10.804020 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.803996 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-scxln"] Sep 10 17:51:10.843020 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.842976 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5"] Sep 10 17:51:10.844362 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:51:10.844335 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7ae526_4baf_4912_bc21_d71d1a447094.slice/crio-bb4dbeea1305d833b326410ac4f479b4b8132163879c27c86bac8edf041a1f86 WatchSource:0}: Error finding container bb4dbeea1305d833b326410ac4f479b4b8132163879c27c86bac8edf041a1f86: Status 404 returned error can't find the container with id bb4dbeea1305d833b326410ac4f479b4b8132163879c27c86bac8edf041a1f86 Sep 10 17:51:10.845703 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.845681 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 17:51:10.959046 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.958950 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgwlw\" (UniqueName: \"kubernetes.io/projected/38bd9222-ee32-4d1e-810f-2c5116b874b3-kube-api-access-fgwlw\") pod \"payload-pre-processing-59fbc47bf8-scxln\" (UID: \"38bd9222-ee32-4d1e-810f-2c5116b874b3\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-scxln" Sep 10 17:51:10.959213 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:10.959057 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38bd9222-ee32-4d1e-810f-2c5116b874b3-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-scxln\" (UID: \"38bd9222-ee32-4d1e-810f-2c5116b874b3\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-scxln" Sep 10 17:51:11.060511 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:11.060469 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38bd9222-ee32-4d1e-810f-2c5116b874b3-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-scxln\" (UID: \"38bd9222-ee32-4d1e-810f-2c5116b874b3\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-scxln" Sep 10 17:51:11.060729 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:11.060559 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgwlw\" (UniqueName: \"kubernetes.io/projected/38bd9222-ee32-4d1e-810f-2c5116b874b3-kube-api-access-fgwlw\") pod \"payload-pre-processing-59fbc47bf8-scxln\" (UID: \"38bd9222-ee32-4d1e-810f-2c5116b874b3\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-scxln" Sep 10 17:51:11.061323 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:11.061289 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38bd9222-ee32-4d1e-810f-2c5116b874b3-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-scxln\" (UID: \"38bd9222-ee32-4d1e-810f-2c5116b874b3\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-scxln" Sep 10 17:51:11.069188 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:11.069160 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgwlw\" (UniqueName: \"kubernetes.io/projected/38bd9222-ee32-4d1e-810f-2c5116b874b3-kube-api-access-fgwlw\") pod \"payload-pre-processing-59fbc47bf8-scxln\" (UID: \"38bd9222-ee32-4d1e-810f-2c5116b874b3\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-scxln" Sep 10 17:51:11.099470 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:11.099440 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-scxln" Sep 10 17:51:11.367910 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:11.367807 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-scxln"] Sep 10 17:51:11.372472 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:51:11.372440 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38bd9222_ee32_4d1e_810f_2c5116b874b3.slice/crio-3efe548c9441641fcb67d2df7c0de1af017570f038921901cc1e717ef411c08b WatchSource:0}: Error finding container 3efe548c9441641fcb67d2df7c0de1af017570f038921901cc1e717ef411c08b: Status 404 returned error can't find the container with id 3efe548c9441641fcb67d2df7c0de1af017570f038921901cc1e717ef411c08b Sep 10 17:51:11.470638 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:11.470605 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" event={"ID":"3e7ae526-4baf-4912-bc21-d71d1a447094","Type":"ContainerStarted","Data":"bb4dbeea1305d833b326410ac4f479b4b8132163879c27c86bac8edf041a1f86"} Sep 10 17:51:11.471855 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:11.471827 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-scxln" event={"ID":"38bd9222-ee32-4d1e-810f-2c5116b874b3","Type":"ContainerStarted","Data":"3efe548c9441641fcb67d2df7c0de1af017570f038921901cc1e717ef411c08b"} Sep 10 17:51:13.485559 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:13.485520 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" event={"ID":"3e7ae526-4baf-4912-bc21-d71d1a447094","Type":"ContainerStarted","Data":"8691d33f3077e49b53ce57195ffa952f449d38c226a801f07d18811b4ca351ae"} Sep 10 17:51:13.486032 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:13.485669 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:13.487063 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:13.487040 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-scxln" event={"ID":"38bd9222-ee32-4d1e-810f-2c5116b874b3","Type":"ContainerStarted","Data":"bcd310f2644aea90fdad69218ddc8cd0751c07029c0d765a64fa463254253827"} Sep 10 17:51:13.487175 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:13.487163 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-scxln" Sep 10 17:51:13.503281 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:13.503238 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" podStartSLOduration=1.737498339 podStartE2EDuration="3.503225478s" totalImagesPullingTime="1.765727139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:51:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:51:13.502462996 +0000 UTC m=+677.566873715" watchObservedRunningTime="2026-09-10 17:51:13.503225478 +0000 UTC m=+677.567636224" Sep 10 17:51:13.517627 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:13.517579 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-scxln" podStartSLOduration=1.897783319 podStartE2EDuration="3.517564173s" totalImagesPullingTime="1.619780854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:51:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:51:13.516801918 +0000 UTC m=+677.581212638" watchObservedRunningTime="2026-09-10 17:51:13.517564173 +0000 UTC m=+677.581974882" Sep 10 17:51:19.497422 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:19.497393 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f975847c4-9f7c5" Sep 10 17:51:20.547627 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:20.547583 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:20.643189 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:20.643155 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:24.494319 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:24.494289 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-scxln" Sep 10 17:51:29.227840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.227797 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m"] Sep 10 17:51:29.231278 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.231259 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.233953 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.233923 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 17:51:29.234295 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.234274 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 17:51:29.234507 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.234483 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-g6mxf\"" Sep 10 17:51:29.234614 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.234591 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 10 17:51:29.240265 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.240240 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m"] Sep 10 17:51:29.270703 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.270675 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:29.306071 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.306043 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/994a7880-c9d3-4793-bc0a-9d23466d242f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.306217 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.306081 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/994a7880-c9d3-4793-bc0a-9d23466d242f-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.306217 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.306106 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gppg8\" (UniqueName: \"kubernetes.io/projected/994a7880-c9d3-4793-bc0a-9d23466d242f-kube-api-access-gppg8\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.306217 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.306186 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/994a7880-c9d3-4793-bc0a-9d23466d242f-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.306320 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.306215 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/994a7880-c9d3-4793-bc0a-9d23466d242f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.306320 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.306282 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/994a7880-c9d3-4793-bc0a-9d23466d242f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.407372 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.407338 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/994a7880-c9d3-4793-bc0a-9d23466d242f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.407577 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.407396 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/994a7880-c9d3-4793-bc0a-9d23466d242f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.407577 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.407442 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/994a7880-c9d3-4793-bc0a-9d23466d242f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.407577 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.407462 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/994a7880-c9d3-4793-bc0a-9d23466d242f-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.407577 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.407478 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gppg8\" (UniqueName: \"kubernetes.io/projected/994a7880-c9d3-4793-bc0a-9d23466d242f-kube-api-access-gppg8\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.407577 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.407524 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/994a7880-c9d3-4793-bc0a-9d23466d242f-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.407907 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.407867 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/994a7880-c9d3-4793-bc0a-9d23466d242f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.407994 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.407953 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/994a7880-c9d3-4793-bc0a-9d23466d242f-home\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.408044 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.408014 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/994a7880-c9d3-4793-bc0a-9d23466d242f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.409690 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.409671 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/994a7880-c9d3-4793-bc0a-9d23466d242f-dshm\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.410009 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.409971 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/994a7880-c9d3-4793-bc0a-9d23466d242f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.415423 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.415402 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gppg8\" (UniqueName: \"kubernetes.io/projected/994a7880-c9d3-4793-bc0a-9d23466d242f-kube-api-access-gppg8\") pod \"e2e-distinct-2-simulated-kserve-859856b874-zq62m\" (UID: \"994a7880-c9d3-4793-bc0a-9d23466d242f\") " pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.543492 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.543398 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:29.814794 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:29.814760 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m"] Sep 10 17:51:29.816947 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:51:29.816918 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod994a7880_c9d3_4793_bc0a_9d23466d242f.slice/crio-914820445b5c8cf56dc294ceb08e897032b461903d7345c59fac5b62169ee54e WatchSource:0}: Error finding container 914820445b5c8cf56dc294ceb08e897032b461903d7345c59fac5b62169ee54e: Status 404 returned error can't find the container with id 914820445b5c8cf56dc294ceb08e897032b461903d7345c59fac5b62169ee54e Sep 10 17:51:30.571346 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:30.571304 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" event={"ID":"994a7880-c9d3-4793-bc0a-9d23466d242f","Type":"ContainerStarted","Data":"914820445b5c8cf56dc294ceb08e897032b461903d7345c59fac5b62169ee54e"} Sep 10 17:51:31.577387 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:31.577350 2565 generic.go:356] "Generic (PLEG): container finished" podID="994a7880-c9d3-4793-bc0a-9d23466d242f" containerID="a38118e36e42c33b74d620363aab6966a2804b0d7ab3d825bd58502a9e2e41a0" exitCode=0 Sep 10 17:51:31.577833 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:31.577432 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" event={"ID":"994a7880-c9d3-4793-bc0a-9d23466d242f","Type":"ContainerDied","Data":"a38118e36e42c33b74d620363aab6966a2804b0d7ab3d825bd58502a9e2e41a0"} Sep 10 17:51:31.577833 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:31.577746 2565 scope.go:117] "RemoveContainer" containerID="a38118e36e42c33b74d620363aab6966a2804b0d7ab3d825bd58502a9e2e41a0" Sep 10 17:51:32.585109 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:32.585069 2565 generic.go:356] "Generic (PLEG): container finished" podID="994a7880-c9d3-4793-bc0a-9d23466d242f" containerID="6a37df01e80a38697dcd4ac891ab08e091f6de10135d5d93ecc84ef9ff9fdfe3" exitCode=0 Sep 10 17:51:32.585606 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:32.585157 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" event={"ID":"994a7880-c9d3-4793-bc0a-9d23466d242f","Type":"ContainerDied","Data":"6a37df01e80a38697dcd4ac891ab08e091f6de10135d5d93ecc84ef9ff9fdfe3"} Sep 10 17:51:32.585606 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:32.585224 2565 scope.go:117] "RemoveContainer" containerID="a38118e36e42c33b74d620363aab6966a2804b0d7ab3d825bd58502a9e2e41a0" Sep 10 17:51:32.585606 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:32.585474 2565 scope.go:117] "RemoveContainer" containerID="6a37df01e80a38697dcd4ac891ab08e091f6de10135d5d93ecc84ef9ff9fdfe3" Sep 10 17:51:32.585732 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:51:32.585669 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:51:33.592101 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:33.592067 2565 scope.go:117] "RemoveContainer" containerID="6a37df01e80a38697dcd4ac891ab08e091f6de10135d5d93ecc84ef9ff9fdfe3" Sep 10 17:51:33.592468 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:51:33.592255 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:51:34.452391 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:34.452354 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:39.543995 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:39.543949 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:39.544481 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:39.544023 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:39.544481 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:39.544039 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:39.544481 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:39.544051 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:39.544481 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:39.544392 2565 scope.go:117] "RemoveContainer" containerID="6a37df01e80a38697dcd4ac891ab08e091f6de10135d5d93ecc84ef9ff9fdfe3" Sep 10 17:51:39.544633 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:51:39.544550 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:51:45.305028 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.304995 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb"] Sep 10 17:51:45.310900 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.310865 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.313234 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.313213 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Sep 10 17:51:45.317907 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.317866 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb"] Sep 10 17:51:45.346124 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.346096 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:45.351776 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.351744 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79a24d23-109e-4c38-b190-882d8c4b29ea-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.351928 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.351817 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79a24d23-109e-4c38-b190-882d8c4b29ea-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.351928 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.351846 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79a24d23-109e-4c38-b190-882d8c4b29ea-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.351928 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.351910 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pskbz\" (UniqueName: \"kubernetes.io/projected/79a24d23-109e-4c38-b190-882d8c4b29ea-kube-api-access-pskbz\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.352093 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.351941 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79a24d23-109e-4c38-b190-882d8c4b29ea-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.352093 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.351982 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79a24d23-109e-4c38-b190-882d8c4b29ea-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.453534 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.453496 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79a24d23-109e-4c38-b190-882d8c4b29ea-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.453700 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.453570 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79a24d23-109e-4c38-b190-882d8c4b29ea-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.453700 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.453597 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79a24d23-109e-4c38-b190-882d8c4b29ea-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.453700 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.453633 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pskbz\" (UniqueName: \"kubernetes.io/projected/79a24d23-109e-4c38-b190-882d8c4b29ea-kube-api-access-pskbz\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.453700 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.453665 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79a24d23-109e-4c38-b190-882d8c4b29ea-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.453924 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.453704 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79a24d23-109e-4c38-b190-882d8c4b29ea-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.454118 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.454087 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79a24d23-109e-4c38-b190-882d8c4b29ea-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.454225 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.454159 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79a24d23-109e-4c38-b190-882d8c4b29ea-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.454272 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.454226 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79a24d23-109e-4c38-b190-882d8c4b29ea-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.455719 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.455692 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79a24d23-109e-4c38-b190-882d8c4b29ea-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.456158 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.456141 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79a24d23-109e-4c38-b190-882d8c4b29ea-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.462117 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.462070 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pskbz\" (UniqueName: \"kubernetes.io/projected/79a24d23-109e-4c38-b190-882d8c4b29ea-kube-api-access-pskbz\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb\" (UID: \"79a24d23-109e-4c38-b190-882d8c4b29ea\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.625455 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.625369 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:45.900381 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:45.900354 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb"] Sep 10 17:51:45.902252 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:51:45.902224 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79a24d23_109e_4c38_b190_882d8c4b29ea.slice/crio-aa811a8ce7770c3a2c36a0538ed0f8af87ded6308ce33a1f2ed288fc38df89e7 WatchSource:0}: Error finding container aa811a8ce7770c3a2c36a0538ed0f8af87ded6308ce33a1f2ed288fc38df89e7: Status 404 returned error can't find the container with id aa811a8ce7770c3a2c36a0538ed0f8af87ded6308ce33a1f2ed288fc38df89e7 Sep 10 17:51:46.657414 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:46.657373 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" event={"ID":"79a24d23-109e-4c38-b190-882d8c4b29ea","Type":"ContainerStarted","Data":"aa811a8ce7770c3a2c36a0538ed0f8af87ded6308ce33a1f2ed288fc38df89e7"} Sep 10 17:51:47.664086 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:47.664051 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" event={"ID":"79a24d23-109e-4c38-b190-882d8c4b29ea","Type":"ContainerStarted","Data":"74803e84f993db322d3c103da8a4270357b5657152905b5939b238872e3ed26e"} Sep 10 17:51:47.680930 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:47.680863 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" podStartSLOduration=1.878264597 podStartE2EDuration="2.680845218s" totalImagesPullingTime="802.580621ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:51:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:51:47.680395528 +0000 UTC m=+711.744806264" watchObservedRunningTime="2026-09-10 17:51:47.680845218 +0000 UTC m=+711.745255929" Sep 10 17:51:53.401742 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:53.401662 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:51:54.432477 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:54.432445 2565 scope.go:117] "RemoveContainer" containerID="6a37df01e80a38697dcd4ac891ab08e091f6de10135d5d93ecc84ef9ff9fdfe3" Sep 10 17:51:55.626228 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:55.626185 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:55.626601 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:55.626250 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:55.638104 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:55.638081 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:55.703607 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:55.703573 2565 generic.go:356] "Generic (PLEG): container finished" podID="994a7880-c9d3-4793-bc0a-9d23466d242f" containerID="3ec3751da364af6825ba3d01859c8592cd9c5cdb0111df7cb9143ccf8a8c0f51" exitCode=0 Sep 10 17:51:55.703760 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:55.703645 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" event={"ID":"994a7880-c9d3-4793-bc0a-9d23466d242f","Type":"ContainerDied","Data":"3ec3751da364af6825ba3d01859c8592cd9c5cdb0111df7cb9143ccf8a8c0f51"} Sep 10 17:51:55.703760 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:55.703699 2565 scope.go:117] "RemoveContainer" containerID="6a37df01e80a38697dcd4ac891ab08e091f6de10135d5d93ecc84ef9ff9fdfe3" Sep 10 17:51:55.704237 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:55.704213 2565 scope.go:117] "RemoveContainer" containerID="3ec3751da364af6825ba3d01859c8592cd9c5cdb0111df7cb9143ccf8a8c0f51" Sep 10 17:51:55.704428 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:51:55.704391 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:51:55.715108 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:55.715087 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb" Sep 10 17:51:59.543629 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:59.543599 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:59.544029 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:59.543651 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:59.544029 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:59.543666 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:59.544029 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:59.543681 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:51:59.544135 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:51:59.544034 2565 scope.go:117] "RemoveContainer" containerID="3ec3751da364af6825ba3d01859c8592cd9c5cdb0111df7cb9143ccf8a8c0f51" Sep 10 17:51:59.544217 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:51:59.544196 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:52:06.260728 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:06.260682 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:12.431983 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:12.431951 2565 scope.go:117] "RemoveContainer" containerID="3ec3751da364af6825ba3d01859c8592cd9c5cdb0111df7cb9143ccf8a8c0f51" Sep 10 17:52:12.432455 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:52:12.432115 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:52:13.818121 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:13.818087 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n"] Sep 10 17:52:13.823492 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:13.823470 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:13.825702 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:13.825671 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 10 17:52:13.832589 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:13.832568 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n"] Sep 10 17:52:13.854954 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:13.854921 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:14.011259 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.011220 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/216d48af-954d-422e-b4ca-662eb3a9dd65-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.011445 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.011282 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/216d48af-954d-422e-b4ca-662eb3a9dd65-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.011445 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.011383 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/216d48af-954d-422e-b4ca-662eb3a9dd65-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.011445 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.011420 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8jzx\" (UniqueName: \"kubernetes.io/projected/216d48af-954d-422e-b4ca-662eb3a9dd65-kube-api-access-g8jzx\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.011598 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.011449 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/216d48af-954d-422e-b4ca-662eb3a9dd65-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.011598 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.011486 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/216d48af-954d-422e-b4ca-662eb3a9dd65-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.112840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.112758 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/216d48af-954d-422e-b4ca-662eb3a9dd65-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.112840 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.112807 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/216d48af-954d-422e-b4ca-662eb3a9dd65-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.113058 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.112988 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/216d48af-954d-422e-b4ca-662eb3a9dd65-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.113058 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.113023 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8jzx\" (UniqueName: \"kubernetes.io/projected/216d48af-954d-422e-b4ca-662eb3a9dd65-kube-api-access-g8jzx\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.113172 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.113059 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/216d48af-954d-422e-b4ca-662eb3a9dd65-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.113172 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.113090 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/216d48af-954d-422e-b4ca-662eb3a9dd65-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.113285 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.113195 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/216d48af-954d-422e-b4ca-662eb3a9dd65-model-cache\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.113285 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.113256 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/216d48af-954d-422e-b4ca-662eb3a9dd65-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.113414 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.113397 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/216d48af-954d-422e-b4ca-662eb3a9dd65-home\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.115131 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.115108 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/216d48af-954d-422e-b4ca-662eb3a9dd65-dshm\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.115484 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.115464 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/216d48af-954d-422e-b4ca-662eb3a9dd65-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.121960 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.121940 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8jzx\" (UniqueName: \"kubernetes.io/projected/216d48af-954d-422e-b4ca-662eb3a9dd65-kube-api-access-g8jzx\") pod \"facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n\" (UID: \"216d48af-954d-422e-b4ca-662eb3a9dd65\") " pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.135815 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.135794 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:14.411758 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.411731 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n"] Sep 10 17:52:14.413861 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:52:14.413837 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216d48af_954d_422e_b4ca_662eb3a9dd65.slice/crio-723df355d3e019cfafddbeab4bdc47295a5d489c3531297f4705fc936ebdf757 WatchSource:0}: Error finding container 723df355d3e019cfafddbeab4bdc47295a5d489c3531297f4705fc936ebdf757: Status 404 returned error can't find the container with id 723df355d3e019cfafddbeab4bdc47295a5d489c3531297f4705fc936ebdf757 Sep 10 17:52:14.801838 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.801809 2565 generic.go:356] "Generic (PLEG): container finished" podID="216d48af-954d-422e-b4ca-662eb3a9dd65" containerID="289d43af758168a55513e7a13fe120c056891c51b3bbca073c75381627f7dd6f" exitCode=0 Sep 10 17:52:14.802007 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.801919 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" event={"ID":"216d48af-954d-422e-b4ca-662eb3a9dd65","Type":"ContainerDied","Data":"289d43af758168a55513e7a13fe120c056891c51b3bbca073c75381627f7dd6f"} Sep 10 17:52:14.802007 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.801975 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" event={"ID":"216d48af-954d-422e-b4ca-662eb3a9dd65","Type":"ContainerStarted","Data":"723df355d3e019cfafddbeab4bdc47295a5d489c3531297f4705fc936ebdf757"} Sep 10 17:52:14.802114 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:14.802100 2565 scope.go:117] "RemoveContainer" containerID="289d43af758168a55513e7a13fe120c056891c51b3bbca073c75381627f7dd6f" Sep 10 17:52:15.782610 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:15.782580 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:15.809164 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:15.809130 2565 generic.go:356] "Generic (PLEG): container finished" podID="216d48af-954d-422e-b4ca-662eb3a9dd65" containerID="bb2538c2c2ba3aa8376006f5d467c8619bf45035295a7e5e326dccd5038da4c6" exitCode=0 Sep 10 17:52:15.809313 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:15.809178 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" event={"ID":"216d48af-954d-422e-b4ca-662eb3a9dd65","Type":"ContainerDied","Data":"bb2538c2c2ba3aa8376006f5d467c8619bf45035295a7e5e326dccd5038da4c6"} Sep 10 17:52:15.809313 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:15.809265 2565 scope.go:117] "RemoveContainer" containerID="289d43af758168a55513e7a13fe120c056891c51b3bbca073c75381627f7dd6f" Sep 10 17:52:15.809507 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:15.809492 2565 scope.go:117] "RemoveContainer" containerID="bb2538c2c2ba3aa8376006f5d467c8619bf45035295a7e5e326dccd5038da4c6" Sep 10 17:52:15.809703 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:52:15.809681 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:52:15.880663 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:15.880639 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:16.815925 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:16.815897 2565 scope.go:117] "RemoveContainer" containerID="bb2538c2c2ba3aa8376006f5d467c8619bf45035295a7e5e326dccd5038da4c6" Sep 10 17:52:16.816258 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:52:16.816054 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:52:19.387197 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:19.387160 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:23.340011 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:23.339975 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:24.135991 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:24.135880 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:24.136142 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:24.136013 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:24.136142 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:24.136028 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:24.136142 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:24.136041 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:24.136394 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:24.136383 2565 scope.go:117] "RemoveContainer" containerID="bb2538c2c2ba3aa8376006f5d467c8619bf45035295a7e5e326dccd5038da4c6" Sep 10 17:52:24.136564 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:52:24.136547 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:52:25.432667 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:25.432637 2565 scope.go:117] "RemoveContainer" containerID="3ec3751da364af6825ba3d01859c8592cd9c5cdb0111df7cb9143ccf8a8c0f51" Sep 10 17:52:25.863590 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:25.863557 2565 generic.go:356] "Generic (PLEG): container finished" podID="994a7880-c9d3-4793-bc0a-9d23466d242f" containerID="6f82bd9b496bb04528e980598ef0035b1f34ec5482d45bf7daac6329b2bb2ee1" exitCode=0 Sep 10 17:52:25.863743 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:25.863625 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" event={"ID":"994a7880-c9d3-4793-bc0a-9d23466d242f","Type":"ContainerDied","Data":"6f82bd9b496bb04528e980598ef0035b1f34ec5482d45bf7daac6329b2bb2ee1"} Sep 10 17:52:25.863743 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:25.863682 2565 scope.go:117] "RemoveContainer" containerID="3ec3751da364af6825ba3d01859c8592cd9c5cdb0111df7cb9143ccf8a8c0f51" Sep 10 17:52:25.864080 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:25.864064 2565 scope.go:117] "RemoveContainer" containerID="6f82bd9b496bb04528e980598ef0035b1f34ec5482d45bf7daac6329b2bb2ee1" Sep 10 17:52:25.864317 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:52:25.864293 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:52:29.543919 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:29.543869 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:52:29.544274 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:29.543936 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:52:29.544274 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:29.543963 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:52:29.544274 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:29.543980 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:52:29.544401 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:29.544299 2565 scope.go:117] "RemoveContainer" containerID="6f82bd9b496bb04528e980598ef0035b1f34ec5482d45bf7daac6329b2bb2ee1" Sep 10 17:52:29.544505 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:52:29.544482 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:52:37.432124 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:37.432089 2565 scope.go:117] "RemoveContainer" containerID="bb2538c2c2ba3aa8376006f5d467c8619bf45035295a7e5e326dccd5038da4c6" Sep 10 17:52:37.928922 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:37.928875 2565 generic.go:356] "Generic (PLEG): container finished" podID="216d48af-954d-422e-b4ca-662eb3a9dd65" containerID="64986c2100892dac69164998843d4bedbc04d4ab1e6e96778753dabf30f260dc" exitCode=0 Sep 10 17:52:37.929096 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:37.928952 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" event={"ID":"216d48af-954d-422e-b4ca-662eb3a9dd65","Type":"ContainerDied","Data":"64986c2100892dac69164998843d4bedbc04d4ab1e6e96778753dabf30f260dc"} Sep 10 17:52:37.929096 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:37.929026 2565 scope.go:117] "RemoveContainer" containerID="bb2538c2c2ba3aa8376006f5d467c8619bf45035295a7e5e326dccd5038da4c6" Sep 10 17:52:37.929464 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:37.929443 2565 scope.go:117] "RemoveContainer" containerID="64986c2100892dac69164998843d4bedbc04d4ab1e6e96778753dabf30f260dc" Sep 10 17:52:37.929667 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:52:37.929648 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:52:42.432813 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:42.432777 2565 scope.go:117] "RemoveContainer" containerID="6f82bd9b496bb04528e980598ef0035b1f34ec5482d45bf7daac6329b2bb2ee1" Sep 10 17:52:42.433210 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:52:42.433045 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:52:43.144041 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:43.144005 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:43.206038 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:43.206010 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:52:44.136839 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:44.136801 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:44.137334 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:44.136863 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:44.137334 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:44.136878 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:44.137334 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:44.136904 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:52:44.137334 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:44.137238 2565 scope.go:117] "RemoveContainer" containerID="64986c2100892dac69164998843d4bedbc04d4ab1e6e96778753dabf30f260dc" Sep 10 17:52:44.137474 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:52:44.137398 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:52:57.432395 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:57.432365 2565 scope.go:117] "RemoveContainer" containerID="6f82bd9b496bb04528e980598ef0035b1f34ec5482d45bf7daac6329b2bb2ee1" Sep 10 17:52:57.432767 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:52:57.432549 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:52:59.431693 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:52:59.431657 2565 scope.go:117] "RemoveContainer" containerID="64986c2100892dac69164998843d4bedbc04d4ab1e6e96778753dabf30f260dc" Sep 10 17:53:00.039605 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:00.039503 2565 generic.go:356] "Generic (PLEG): container finished" podID="216d48af-954d-422e-b4ca-662eb3a9dd65" containerID="78be0bdb3a170fee031d453e6aa2b89a8d04738e9591038f0d28f31d180f4d18" exitCode=0 Sep 10 17:53:00.039605 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:00.039585 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" event={"ID":"216d48af-954d-422e-b4ca-662eb3a9dd65","Type":"ContainerDied","Data":"78be0bdb3a170fee031d453e6aa2b89a8d04738e9591038f0d28f31d180f4d18"} Sep 10 17:53:00.039834 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:00.039678 2565 scope.go:117] "RemoveContainer" containerID="64986c2100892dac69164998843d4bedbc04d4ab1e6e96778753dabf30f260dc" Sep 10 17:53:00.040084 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:00.040060 2565 scope.go:117] "RemoveContainer" containerID="78be0bdb3a170fee031d453e6aa2b89a8d04738e9591038f0d28f31d180f4d18" Sep 10 17:53:00.040294 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:53:00.040275 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:53:04.136282 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:04.136227 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:53:04.136764 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:04.136343 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:53:04.136764 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:04.136367 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:53:04.136764 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:04.136388 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:53:04.136958 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:04.136787 2565 scope.go:117] "RemoveContainer" containerID="78be0bdb3a170fee031d453e6aa2b89a8d04738e9591038f0d28f31d180f4d18" Sep 10 17:53:04.137053 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:53:04.137030 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:53:11.431810 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:11.431779 2565 scope.go:117] "RemoveContainer" containerID="6f82bd9b496bb04528e980598ef0035b1f34ec5482d45bf7daac6329b2bb2ee1" Sep 10 17:53:12.101030 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:12.100944 2565 generic.go:356] "Generic (PLEG): container finished" podID="994a7880-c9d3-4793-bc0a-9d23466d242f" containerID="31ed03730b91a0107ad929fad6d0dbdfb54d4db4e23a0dd4ae86c842f3aef315" exitCode=0 Sep 10 17:53:12.101030 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:12.101014 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" event={"ID":"994a7880-c9d3-4793-bc0a-9d23466d242f","Type":"ContainerDied","Data":"31ed03730b91a0107ad929fad6d0dbdfb54d4db4e23a0dd4ae86c842f3aef315"} Sep 10 17:53:12.101217 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:12.101076 2565 scope.go:117] "RemoveContainer" containerID="6f82bd9b496bb04528e980598ef0035b1f34ec5482d45bf7daac6329b2bb2ee1" Sep 10 17:53:12.101390 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:12.101377 2565 scope.go:117] "RemoveContainer" containerID="31ed03730b91a0107ad929fad6d0dbdfb54d4db4e23a0dd4ae86c842f3aef315" Sep 10 17:53:12.101579 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:53:12.101561 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:53:17.432147 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:17.432116 2565 scope.go:117] "RemoveContainer" containerID="78be0bdb3a170fee031d453e6aa2b89a8d04738e9591038f0d28f31d180f4d18" Sep 10 17:53:17.432616 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:53:17.432264 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:53:19.544073 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:19.544000 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:53:19.544073 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:19.544059 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:53:19.544459 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:19.544078 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:53:19.544459 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:19.544096 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:53:19.544459 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:19.544450 2565 scope.go:117] "RemoveContainer" containerID="31ed03730b91a0107ad929fad6d0dbdfb54d4db4e23a0dd4ae86c842f3aef315" Sep 10 17:53:19.544625 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:53:19.544605 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:53:30.432343 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:30.432306 2565 scope.go:117] "RemoveContainer" containerID="78be0bdb3a170fee031d453e6aa2b89a8d04738e9591038f0d28f31d180f4d18" Sep 10 17:53:30.432835 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:53:30.432484 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:53:31.431707 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:31.431662 2565 scope.go:117] "RemoveContainer" containerID="31ed03730b91a0107ad929fad6d0dbdfb54d4db4e23a0dd4ae86c842f3aef315" Sep 10 17:53:31.431943 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:53:31.431917 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:53:41.432530 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:41.432493 2565 scope.go:117] "RemoveContainer" containerID="78be0bdb3a170fee031d453e6aa2b89a8d04738e9591038f0d28f31d180f4d18" Sep 10 17:53:42.254130 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:42.254097 2565 generic.go:356] "Generic (PLEG): container finished" podID="216d48af-954d-422e-b4ca-662eb3a9dd65" containerID="4fbb92b290e76970e07c4a980e6b6ca8ba8988dac91ea26d7956d1f59f4ba6fc" exitCode=0 Sep 10 17:53:42.254304 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:42.254156 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" event={"ID":"216d48af-954d-422e-b4ca-662eb3a9dd65","Type":"ContainerDied","Data":"4fbb92b290e76970e07c4a980e6b6ca8ba8988dac91ea26d7956d1f59f4ba6fc"} Sep 10 17:53:42.254304 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:42.254202 2565 scope.go:117] "RemoveContainer" containerID="78be0bdb3a170fee031d453e6aa2b89a8d04738e9591038f0d28f31d180f4d18" Sep 10 17:53:42.254559 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:42.254540 2565 scope.go:117] "RemoveContainer" containerID="4fbb92b290e76970e07c4a980e6b6ca8ba8988dac91ea26d7956d1f59f4ba6fc" Sep 10 17:53:42.254792 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:53:42.254772 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:53:44.136502 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:44.136462 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:53:44.136882 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:44.136527 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:53:44.136882 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:44.136546 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:53:44.136882 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:44.136559 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:53:44.137026 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:44.136941 2565 scope.go:117] "RemoveContainer" containerID="4fbb92b290e76970e07c4a980e6b6ca8ba8988dac91ea26d7956d1f59f4ba6fc" Sep 10 17:53:44.137135 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:53:44.137115 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:53:46.435205 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:46.435167 2565 scope.go:117] "RemoveContainer" containerID="31ed03730b91a0107ad929fad6d0dbdfb54d4db4e23a0dd4ae86c842f3aef315" Sep 10 17:53:46.435579 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:53:46.435418 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:53:55.432394 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:55.432361 2565 scope.go:117] "RemoveContainer" containerID="4fbb92b290e76970e07c4a980e6b6ca8ba8988dac91ea26d7956d1f59f4ba6fc" Sep 10 17:53:55.432871 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:53:55.432546 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:53:59.432482 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:53:59.432446 2565 scope.go:117] "RemoveContainer" containerID="31ed03730b91a0107ad929fad6d0dbdfb54d4db4e23a0dd4ae86c842f3aef315" Sep 10 17:53:59.432873 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:53:59.432677 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:54:06.435761 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:06.435731 2565 scope.go:117] "RemoveContainer" containerID="4fbb92b290e76970e07c4a980e6b6ca8ba8988dac91ea26d7956d1f59f4ba6fc" Sep 10 17:54:06.436231 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:54:06.435931 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:54:10.432105 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:10.432073 2565 scope.go:117] "RemoveContainer" containerID="31ed03730b91a0107ad929fad6d0dbdfb54d4db4e23a0dd4ae86c842f3aef315" Sep 10 17:54:10.432533 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:54:10.432232 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:54:21.431782 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:21.431747 2565 scope.go:117] "RemoveContainer" containerID="4fbb92b290e76970e07c4a980e6b6ca8ba8988dac91ea26d7956d1f59f4ba6fc" Sep 10 17:54:21.432204 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:54:21.431972 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:54:23.432106 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:23.432068 2565 scope.go:117] "RemoveContainer" containerID="31ed03730b91a0107ad929fad6d0dbdfb54d4db4e23a0dd4ae86c842f3aef315" Sep 10 17:54:23.432572 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:54:23.432254 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:54:32.432420 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:32.432382 2565 scope.go:117] "RemoveContainer" containerID="4fbb92b290e76970e07c4a980e6b6ca8ba8988dac91ea26d7956d1f59f4ba6fc" Sep 10 17:54:32.432824 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:54:32.432562 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:54:37.431618 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:37.431585 2565 scope.go:117] "RemoveContainer" containerID="31ed03730b91a0107ad929fad6d0dbdfb54d4db4e23a0dd4ae86c842f3aef315" Sep 10 17:54:38.531633 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:38.531598 2565 generic.go:356] "Generic (PLEG): container finished" podID="994a7880-c9d3-4793-bc0a-9d23466d242f" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" exitCode=0 Sep 10 17:54:38.532085 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:38.531676 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" event={"ID":"994a7880-c9d3-4793-bc0a-9d23466d242f","Type":"ContainerDied","Data":"860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2"} Sep 10 17:54:38.532085 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:38.531767 2565 scope.go:117] "RemoveContainer" containerID="31ed03730b91a0107ad929fad6d0dbdfb54d4db4e23a0dd4ae86c842f3aef315" Sep 10 17:54:38.532190 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:38.532173 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:54:38.532402 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:54:38.532383 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:54:39.543852 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:39.543824 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:54:39.544245 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:39.543867 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:54:39.544245 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:39.543901 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:54:39.544245 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:39.543917 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" Sep 10 17:54:39.544245 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:39.544199 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:54:39.544387 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:54:39.544362 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:54:40.547533 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:40.547505 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:54:40.547920 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:54:40.547689 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:54:45.431760 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:45.431729 2565 scope.go:117] "RemoveContainer" containerID="4fbb92b290e76970e07c4a980e6b6ca8ba8988dac91ea26d7956d1f59f4ba6fc" Sep 10 17:54:45.432156 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:54:45.431913 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:54:53.431971 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:53.431870 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:54:53.432408 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:54:53.432079 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:54:56.411694 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:56.411657 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g2pbs_f558fde8-d070-4538-bb97-19bb59832b85/ovn-acl-logging/0.log" Sep 10 17:54:56.412144 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:56.411661 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g2pbs_f558fde8-d070-4538-bb97-19bb59832b85/ovn-acl-logging/0.log" Sep 10 17:54:59.432031 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:54:59.432003 2565 scope.go:117] "RemoveContainer" containerID="4fbb92b290e76970e07c4a980e6b6ca8ba8988dac91ea26d7956d1f59f4ba6fc" Sep 10 17:54:59.432431 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:54:59.432147 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:55:06.439396 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:06.439364 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:55:06.439795 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:55:06.439532 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:55:12.438158 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:12.438128 2565 scope.go:117] "RemoveContainer" containerID="4fbb92b290e76970e07c4a980e6b6ca8ba8988dac91ea26d7956d1f59f4ba6fc" Sep 10 17:55:12.702813 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:12.702786 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" event={"ID":"216d48af-954d-422e-b4ca-662eb3a9dd65","Type":"ContainerStarted","Data":"11e6eb3fb492ff991b93ce38940af6920320614c579259db1471abdae9398119"} Sep 10 17:55:12.726161 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:12.726112 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podStartSLOduration=178.439402344 podStartE2EDuration="2m59.726096197s" totalImagesPullingTime="1.286693853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:52:13 +0000 UTC" imagePullSessionsCount=6 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:55:12.724688845 +0000 UTC m=+916.789099557" watchObservedRunningTime="2026-09-10 17:55:12.726096197 +0000 UTC m=+916.790506908" Sep 10 17:55:13.710479 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:13.710442 2565 generic.go:356] "Generic (PLEG): container finished" podID="216d48af-954d-422e-b4ca-662eb3a9dd65" containerID="11e6eb3fb492ff991b93ce38940af6920320614c579259db1471abdae9398119" exitCode=0 Sep 10 17:55:13.710931 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:13.710512 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" event={"ID":"216d48af-954d-422e-b4ca-662eb3a9dd65","Type":"ContainerDied","Data":"11e6eb3fb492ff991b93ce38940af6920320614c579259db1471abdae9398119"} Sep 10 17:55:13.710931 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:13.710569 2565 scope.go:117] "RemoveContainer" containerID="4fbb92b290e76970e07c4a980e6b6ca8ba8988dac91ea26d7956d1f59f4ba6fc" Sep 10 17:55:13.710931 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:13.710923 2565 scope.go:117] "RemoveContainer" containerID="11e6eb3fb492ff991b93ce38940af6920320614c579259db1471abdae9398119" Sep 10 17:55:13.711146 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:55:13.711121 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:55:14.135961 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:14.135925 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:55:14.136144 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:14.136030 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:55:14.136144 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:14.136057 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:55:14.136144 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:14.136074 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" Sep 10 17:55:14.717596 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:14.717566 2565 scope.go:117] "RemoveContainer" containerID="11e6eb3fb492ff991b93ce38940af6920320614c579259db1471abdae9398119" Sep 10 17:55:14.717984 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:55:14.717729 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:55:17.431960 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:17.431928 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:55:17.432345 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:55:17.432098 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:55:25.432715 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:25.432681 2565 scope.go:117] "RemoveContainer" containerID="11e6eb3fb492ff991b93ce38940af6920320614c579259db1471abdae9398119" Sep 10 17:55:25.433204 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:55:25.432825 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:55:29.432467 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:29.432435 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:55:29.432847 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:55:29.432624 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:55:40.432359 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:40.432318 2565 scope.go:117] "RemoveContainer" containerID="11e6eb3fb492ff991b93ce38940af6920320614c579259db1471abdae9398119" Sep 10 17:55:40.432870 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:40.432441 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:55:40.432870 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:55:40.432511 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:55:40.432870 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:55:40.432651 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:55:51.431785 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:51.431744 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:55:51.432286 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:55:51.432013 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:55:55.432395 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:55:55.432353 2565 scope.go:117] "RemoveContainer" containerID="11e6eb3fb492ff991b93ce38940af6920320614c579259db1471abdae9398119" Sep 10 17:55:55.432825 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:55:55.432595 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:56:03.431856 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:03.431814 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:56:03.432448 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:56:03.432102 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:56:06.442214 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:06.442183 2565 scope.go:117] "RemoveContainer" containerID="11e6eb3fb492ff991b93ce38940af6920320614c579259db1471abdae9398119" Sep 10 17:56:06.442668 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:56:06.442409 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:56:15.522115 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:15.522072 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:16.433944 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:16.433912 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:56:16.434196 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:56:16.434159 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:56:16.596533 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:16.596487 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:17.431811 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:17.431777 2565 scope.go:117] "RemoveContainer" containerID="11e6eb3fb492ff991b93ce38940af6920320614c579259db1471abdae9398119" Sep 10 17:56:17.432017 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:56:17.431994 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:56:24.432014 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:24.431865 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_216d48af-954d-422e-b4ca-662eb3a9dd65/main/5.log" Sep 10 17:56:24.551940 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:24.551911 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_216d48af-954d-422e-b4ca-662eb3a9dd65/main/5.log" Sep 10 17:56:28.432293 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:28.432246 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:56:28.432687 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:56:28.432493 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:56:29.431987 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:29.431955 2565 scope.go:117] "RemoveContainer" containerID="11e6eb3fb492ff991b93ce38940af6920320614c579259db1471abdae9398119" Sep 10 17:56:29.432161 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:56:29.432102 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:56:31.136023 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:31.135991 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4wwx8_7915f0f9-6e25-48a0-b177-0b406b6c6d87/manager/0.log" Sep 10 17:56:31.258311 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:31.258280 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jj5r8_f179b291-78a4-416a-9950-d04c61a061fc/manager/0.log" Sep 10 17:56:31.385730 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:31.385693 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-864f69dd8b-2s62v_88764722-e6e6-4356-929c-8fb593091a1e/manager/0.log" Sep 10 17:56:31.763805 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:31.763776 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4_60fc9b3c-331c-4a37-ab1a-21fe46eb0066/copy-manifests/0.log" Sep 10 17:56:31.781919 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:31.781876 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4_60fc9b3c-331c-4a37-ab1a-21fe46eb0066/manager/0.log" Sep 10 17:56:32.385093 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:32.385067 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7f975847c4-9f7c5_3e7ae526-4baf-4912-bc21-d71d1a447094/maas-api/0.log" Sep 10 17:56:32.527971 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:32.527937 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-787tw_8627c198-ef2f-4728-aef6-6b64e9592954/postgres/0.log" Sep 10 17:56:33.461321 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:33.461290 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-jb2np_0b932e8b-986e-4d45-870c-9b9b1d12d1f1/manager/0.log" Sep 10 17:56:33.578955 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:33.578925 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-fczpx_e6e60860-6b9f-4969-973a-5e68a1da9400/manager/0.log" Sep 10 17:56:33.691015 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:33.690981 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-5x7xv_803b68d9-477d-40ee-b4a9-12e748a24384/kuadrant-console-plugin/0.log" Sep 10 17:56:33.821273 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:33.821190 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-tp9cd_456cdb24-8990-4e20-95f5-353a0c15ea92/manager/0.log" Sep 10 17:56:34.061628 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:34.061595 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-qbl8m_b8be19ca-3248-451c-bc93-b4eebb5e38bd/manager/0.log" Sep 10 17:56:34.697850 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:34.697820 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f88844dfb-mp78p_6335b441-2450-4b00-b46a-c91a2101a6c5/kube-auth-proxy/0.log" Sep 10 17:56:35.065785 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:35.065698 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-scxln_38bd9222-ee32-4d1e-810f-2c5116b874b3/payload-pre-processing/0.log" Sep 10 17:56:35.655755 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:35.655725 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-859856b874-zq62m_994a7880-c9d3-4793-bc0a-9d23466d242f/main/5.log" Sep 10 17:56:35.890816 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:35.890785 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-5pfkb_79a24d23-109e-4c38-b190-882d8c4b29ea/main/0.log" Sep 10 17:56:36.244022 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:36.243997 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_216d48af-954d-422e-b4ca-662eb3a9dd65/main/5.log" Sep 10 17:56:42.070393 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:42.070359 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:42.167914 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:42.167855 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" limit=1024 Sep 10 17:56:42.431913 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:42.431855 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:56:42.432111 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:56:42.432071 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:56:43.432207 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:43.432178 2565 scope.go:117] "RemoveContainer" containerID="11e6eb3fb492ff991b93ce38940af6920320614c579259db1471abdae9398119" Sep 10 17:56:43.432584 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:56:43.432335 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:56:45.092950 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:45.092918 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-s42t8_af672f7a-0702-4a9b-8a25-5cab8441f3c2/global-pull-secret-syncer/0.log" Sep 10 17:56:45.141610 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:45.141564 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2tz6j_f309e28c-ca0c-40e2-b6b7-3beb2f3213c7/konnectivity-agent/0.log" Sep 10 17:56:45.247601 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:45.247575 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-84.ec2.internal_b0f5917d8bc06c28727df72b1ab49ae9/haproxy/0.log" Sep 10 17:56:49.764901 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:49.764862 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-jb2np_0b932e8b-986e-4d45-870c-9b9b1d12d1f1/manager/0.log" Sep 10 17:56:49.786874 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:49.786844 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-fczpx_e6e60860-6b9f-4969-973a-5e68a1da9400/manager/0.log" Sep 10 17:56:49.810965 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:49.810942 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-5x7xv_803b68d9-477d-40ee-b4a9-12e748a24384/kuadrant-console-plugin/0.log" Sep 10 17:56:49.849339 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:49.849300 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-tp9cd_456cdb24-8990-4e20-95f5-353a0c15ea92/manager/0.log" Sep 10 17:56:49.918415 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:49.918384 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-qbl8m_b8be19ca-3248-451c-bc93-b4eebb5e38bd/manager/0.log" Sep 10 17:56:51.561428 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:51.561396 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-fx8qb_e81e5b8d-0807-48b8-8bf7-d31d799e2d40/cluster-monitoring-operator/0.log" Sep 10 17:56:51.592112 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:51.592081 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-tnvkh_c2c7f5e5-0a1d-4e55-a748-36db237c835b/kube-state-metrics/0.log" Sep 10 17:56:51.608657 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:51.608631 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-tnvkh_c2c7f5e5-0a1d-4e55-a748-36db237c835b/kube-rbac-proxy-main/0.log" Sep 10 17:56:51.628102 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:51.628077 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-tnvkh_c2c7f5e5-0a1d-4e55-a748-36db237c835b/kube-rbac-proxy-self/0.log" Sep 10 17:56:51.682854 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:51.682826 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-gkwr5_b8f58d4b-46b9-4fe9-98dc-3ff4177eb695/monitoring-plugin/0.log" Sep 10 17:56:51.793257 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:51.793229 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jgh7w_c9354ab8-139a-49ea-970f-07013448edac/node-exporter/0.log" Sep 10 17:56:51.816218 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:51.816135 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jgh7w_c9354ab8-139a-49ea-970f-07013448edac/kube-rbac-proxy/0.log" Sep 10 17:56:51.837070 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:51.837045 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jgh7w_c9354ab8-139a-49ea-970f-07013448edac/init-textfile/0.log" Sep 10 17:56:52.365703 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:52.365679 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5854dbc468-jbfxd_452efbb7-5b5e-4b64-8217-cd4e80560ca3/telemeter-client/0.log" Sep 10 17:56:52.394287 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:52.394257 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5854dbc468-jbfxd_452efbb7-5b5e-4b64-8217-cd4e80560ca3/reload/0.log" Sep 10 17:56:52.419153 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:52.419123 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5854dbc468-jbfxd_452efbb7-5b5e-4b64-8217-cd4e80560ca3/kube-rbac-proxy/0.log" Sep 10 17:56:54.432153 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.432124 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:56:54.432541 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:56:54.432315 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:56:54.776384 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.776291 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b"] Sep 10 17:56:54.779874 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.779852 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.782026 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.782009 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qq7nd\"/\"default-dockercfg-9s2j2\"" Sep 10 17:56:54.782697 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.782682 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qq7nd\"/\"openshift-service-ca.crt\"" Sep 10 17:56:54.783945 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.783930 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qq7nd\"/\"kube-root-ca.crt\"" Sep 10 17:56:54.794327 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.794304 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b"] Sep 10 17:56:54.871454 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.871419 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/733717dc-a248-4953-be82-1eb23c797e74-sys\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.871640 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.871483 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/733717dc-a248-4953-be82-1eb23c797e74-podres\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.871640 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.871526 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgnmw\" (UniqueName: \"kubernetes.io/projected/733717dc-a248-4953-be82-1eb23c797e74-kube-api-access-mgnmw\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.871640 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.871553 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/733717dc-a248-4953-be82-1eb23c797e74-proc\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.871640 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.871580 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/733717dc-a248-4953-be82-1eb23c797e74-lib-modules\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.972729 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.972693 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/733717dc-a248-4953-be82-1eb23c797e74-proc\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.972729 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.972734 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/733717dc-a248-4953-be82-1eb23c797e74-lib-modules\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.973013 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.972782 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/733717dc-a248-4953-be82-1eb23c797e74-sys\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.973013 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.972821 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/733717dc-a248-4953-be82-1eb23c797e74-podres\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.973013 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.972831 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/733717dc-a248-4953-be82-1eb23c797e74-proc\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.973013 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.972846 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgnmw\" (UniqueName: \"kubernetes.io/projected/733717dc-a248-4953-be82-1eb23c797e74-kube-api-access-mgnmw\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.973013 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.972968 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/733717dc-a248-4953-be82-1eb23c797e74-sys\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.973013 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.972976 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/733717dc-a248-4953-be82-1eb23c797e74-lib-modules\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.973272 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.973090 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/733717dc-a248-4953-be82-1eb23c797e74-podres\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:54.988291 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:54.988260 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgnmw\" (UniqueName: \"kubernetes.io/projected/733717dc-a248-4953-be82-1eb23c797e74-kube-api-access-mgnmw\") pod \"perf-node-gather-daemonset-nxw8b\" (UID: \"733717dc-a248-4953-be82-1eb23c797e74\") " pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:55.089394 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:55.089298 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:55.372119 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:55.372093 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b"] Sep 10 17:56:55.374732 ip-10-0-130-84 kubenswrapper[2565]: W0910 17:56:55.374711 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod733717dc_a248_4953_be82_1eb23c797e74.slice/crio-cfe4fb61559e4f0d86f4cb791216adb1d97ee757caa7ae7e2b6be439091f0f63 WatchSource:0}: Error finding container cfe4fb61559e4f0d86f4cb791216adb1d97ee757caa7ae7e2b6be439091f0f63: Status 404 returned error can't find the container with id cfe4fb61559e4f0d86f4cb791216adb1d97ee757caa7ae7e2b6be439091f0f63 Sep 10 17:56:55.438158 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:55.432814 2565 scope.go:117] "RemoveContainer" containerID="11e6eb3fb492ff991b93ce38940af6920320614c579259db1471abdae9398119" Sep 10 17:56:55.438158 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:56:55.433204 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:56:55.483662 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:55.483632 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-68699b7576-q6c5r_c879dd63-c950-44b3-ab60-7921c4b728d2/console/0.log" Sep 10 17:56:55.526724 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:55.526691 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-pgwlp_d4f61f63-0da1-4805-9490-f82e8aec75ea/download-server/0.log" Sep 10 17:56:56.210549 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:56.210513 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" event={"ID":"733717dc-a248-4953-be82-1eb23c797e74","Type":"ContainerStarted","Data":"73240172af7d248eb20570a6f27bb7909ced984bdcc9f40ba493e1546cb95476"} Sep 10 17:56:56.210725 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:56.210602 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:56:56.210725 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:56.210620 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" event={"ID":"733717dc-a248-4953-be82-1eb23c797e74","Type":"ContainerStarted","Data":"cfe4fb61559e4f0d86f4cb791216adb1d97ee757caa7ae7e2b6be439091f0f63"} Sep 10 17:56:56.230336 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:56.230288 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" podStartSLOduration=2.230274495 podStartE2EDuration="2.230274495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:56:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:56:56.228287652 +0000 UTC m=+1020.292698382" watchObservedRunningTime="2026-09-10 17:56:56.230274495 +0000 UTC m=+1020.294685239" Sep 10 17:56:57.362472 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:57.362443 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gnq56_25d7cbef-d614-40ec-bf7c-517c5e752cea/dns/0.log" Sep 10 17:56:57.385978 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:57.385954 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gnq56_25d7cbef-d614-40ec-bf7c-517c5e752cea/kube-rbac-proxy/0.log" Sep 10 17:56:57.519358 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:57.519330 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w2k5w_0c6a2e31-247f-4534-845e-8e68feecb206/dns-node-resolver/0.log" Sep 10 17:56:58.051090 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:58.051054 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-54f8fb7c56-74zxh_2f4edcd8-3905-4c92-bfdd-de3e058c21fd/registry/0.log" Sep 10 17:56:58.056938 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:58.056908 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-54f8fb7c56-74zxh_2f4edcd8-3905-4c92-bfdd-de3e058c21fd/registry/1.log" Sep 10 17:56:58.078149 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:58.078124 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-66qw8_dc0c8edc-89e0-4448-896c-78aa56a57d1c/node-ca/0.log" Sep 10 17:56:59.245206 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:59.245178 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f88844dfb-mp78p_6335b441-2450-4b00-b46a-c91a2101a6c5/kube-auth-proxy/0.log" Sep 10 17:56:59.361217 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:56:59.361184 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-scxln_38bd9222-ee32-4d1e-810f-2c5116b874b3/payload-pre-processing/0.log" Sep 10 17:57:00.114281 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:00.114252 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sx5vt_5e2f1a90-af6b-49cc-9466-57b7848dcaf3/serve-healthcheck-canary/0.log" Sep 10 17:57:00.797489 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:00.797461 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w6tmf_326a773c-08de-4f4e-af1e-7c46b8b929c9/kube-rbac-proxy/0.log" Sep 10 17:57:00.826867 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:00.826847 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w6tmf_326a773c-08de-4f4e-af1e-7c46b8b929c9/exporter/0.log" Sep 10 17:57:00.850834 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:00.850793 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w6tmf_326a773c-08de-4f4e-af1e-7c46b8b929c9/extractor/0.log" Sep 10 17:57:02.225643 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:02.225613 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qq7nd/perf-node-gather-daemonset-nxw8b" Sep 10 17:57:02.993410 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:02.993364 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4wwx8_7915f0f9-6e25-48a0-b177-0b406b6c6d87/manager/0.log" Sep 10 17:57:03.037743 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:03.037720 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jj5r8_f179b291-78a4-416a-9950-d04c61a061fc/manager/0.log" Sep 10 17:57:03.081179 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:03.081148 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-864f69dd8b-2s62v_88764722-e6e6-4356-929c-8fb593091a1e/manager/0.log" Sep 10 17:57:03.280562 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:03.280473 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4_60fc9b3c-331c-4a37-ab1a-21fe46eb0066/manager/0.log" Sep 10 17:57:03.324697 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:03.324674 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f7d5cf7ff-94tx4_60fc9b3c-331c-4a37-ab1a-21fe46eb0066/copy-manifests/0.log" Sep 10 17:57:05.257451 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:05.257421 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f88844dfb-mp78p_6335b441-2450-4b00-b46a-c91a2101a6c5/kube-auth-proxy/0.log" Sep 10 17:57:05.310527 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:05.310499 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-scxln_38bd9222-ee32-4d1e-810f-2c5116b874b3/payload-pre-processing/0.log" Sep 10 17:57:06.434297 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:06.434259 2565 scope.go:117] "RemoveContainer" containerID="11e6eb3fb492ff991b93ce38940af6920320614c579259db1471abdae9398119" Sep 10 17:57:06.434712 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:57:06.434493 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n_llm(216d48af-954d-422e-b4ca-662eb3a9dd65)\"" pod="llm/facebook-opt-125m-simulated-kserve-64dfc7b8f-4sp4n" podUID="216d48af-954d-422e-b4ca-662eb3a9dd65" Sep 10 17:57:06.554563 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:06.554533 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f88844dfb-mp78p_6335b441-2450-4b00-b46a-c91a2101a6c5/kube-auth-proxy/0.log" Sep 10 17:57:06.623193 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:06.623167 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-scxln_38bd9222-ee32-4d1e-810f-2c5116b874b3/payload-pre-processing/0.log" Sep 10 17:57:09.432319 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:09.432284 2565 scope.go:117] "RemoveContainer" containerID="860f8984541046af2229b8851d35fe8889eb795f8c6aae428ece13af79f5a3e2" Sep 10 17:57:09.432697 ip-10-0-130-84 kubenswrapper[2565]: E0910 17:57:09.432485 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-2-simulated-kserve-859856b874-zq62m_llm(994a7880-c9d3-4793-bc0a-9d23466d242f)\"" pod="llm/e2e-distinct-2-simulated-kserve-859856b874-zq62m" podUID="994a7880-c9d3-4793-bc0a-9d23466d242f" Sep 10 17:57:10.140633 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:10.140603 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-pkfvp_8a8605cf-a1c3-4608-a2e5-e26aad1444d5/migrator/0.log" Sep 10 17:57:10.163922 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:10.163876 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-pkfvp_8a8605cf-a1c3-4608-a2e5-e26aad1444d5/graceful-termination/0.log" Sep 10 17:57:10.519264 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:10.519187 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-7tr6s_06559b57-0c49-4988-87ab-01fde08339dd/kube-storage-version-migrator-operator/1.log" Sep 10 17:57:10.520930 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:10.520907 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-7tr6s_06559b57-0c49-4988-87ab-01fde08339dd/kube-storage-version-migrator-operator/0.log" Sep 10 17:57:11.854413 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:11.854382 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r4tz2_e3708abb-21c9-4590-90a9-00cfec61f382/kube-multus-additional-cni-plugins/0.log" Sep 10 17:57:11.875383 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:11.875355 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r4tz2_e3708abb-21c9-4590-90a9-00cfec61f382/egress-router-binary-copy/0.log" Sep 10 17:57:11.900591 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:11.900563 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r4tz2_e3708abb-21c9-4590-90a9-00cfec61f382/cni-plugins/0.log" Sep 10 17:57:11.920097 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:11.920069 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r4tz2_e3708abb-21c9-4590-90a9-00cfec61f382/bond-cni-plugin/0.log" Sep 10 17:57:11.940117 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:11.940075 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r4tz2_e3708abb-21c9-4590-90a9-00cfec61f382/routeoverride-cni/0.log" Sep 10 17:57:11.960075 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:11.960053 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r4tz2_e3708abb-21c9-4590-90a9-00cfec61f382/whereabouts-cni-bincopy/0.log" Sep 10 17:57:11.980416 ip-10-0-130-84 kubenswrapper[2565]: I0910 17:57:11.980385 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r4tz2_e3708abb-21c9-4590-90a9-00cfec61f382/whereabouts-cni/0.log"